Hey welcome!
You don't need to rush into fandom, because fandom will bombard you once you stick your neck anywhere close to it
Just remember this: If it exists, there is a Touhou parody of it
So take your time
So you said that you have absolutely no clue where to start right?
Games-wise, you'll want to be looking for one of the windows game on listed on the
front pageI don't know how many of the games you plan on getting at first, but there's several ways of going around playing them
Some people just go in numerical order, starting from 6 and going to 13
But because the games are kind of like separate stories (although there is a continuity), you don't really need to play them in order to understand what's going on. Well, the only exceptions are the last 3 games (11, 12, and 13, which loosely references events/characters started in 10), and the playable characters list tend to borrow characters from past games at times.
If you're going for story, well, there isn't much as you might soon find out. But if you want to go for character introductions and getting to know who's who (which can be helpful) then playing in order might be nice (although even then, all you get in game is several lines of dialogue. If you want backstory and the such, you're going to have to look in the bios). If you're going for gameplay and this is your first danmaku game, then you might want to start off with some of the easier games in the series (#7 or #8 come to mind)
Just for extra info, the games with a decimal after the end are canon, but not part of the 'main series', so you'll probably want to start off with the integer games. Also, #9 is quiet different in gameplay.
This may or may not have been entirely useful info, but enjoy.