Speaking as someone who is planning a Manly Brouhou Reimu costume for Otakon 2010 (I have chest hair and a guy's build, so if I cannot be a trap, I might as well be MASCULINE) I wouldn't oppose more guys in Touhou. We get it, the girls' outfits are pretty to look at and they're cute eye candy. It's easy for outsiders to read "Eww, pedo" and that's where suppressing your power level comes in.
If you're running around the street in a Yukkuri T-shirt, waving a gohei you made out of a twig and Superglued paperboard, shouting "MARISA STOLE THE PRECIOUS THING" while blazing U.N. Owen on your boombox and danmakuing with pebbles, then you deserve to have people think you're creepy. If you're a total hikki who makes no attempt to have human interaction other than printed fanart of your favorite Touhou, then you deserve to have people think you're some fanboy. If you're a denizen of the Internet who just enjoys Touhou and the mythos/characters/music/game, then you're already on Poolswimmer or MotK and guess what? You just enjoy the mythos/characters/music/game. You can like whatever you like but if you're going to be an outwardly forceful douchebag about it (e.g. "TOUHOU IS GREAT AND STUPID WESTERN GAMES SUCK, SUWAKO IS MAI WAIFU IRL" or, turning things around, "YIFF YIFF YIFF HAY GUYS COME CHECK OUT MY FURSONA I DREW ART OF IT HAVING SEX WITH A UNICORNOCEROUS THAT HAS SONIC THE HEDGEHOG'S SNEAKERS AND HAIR") then you're not suppressing your power level, you're being a shitty example for the rest of us, but in the end, do we care what the rest of the world thinks?
That's pretty much all there is to it. If you are any of those and you feel bad because some random schmucks on the Internet think you're pedo, you should get the hell off the Internet right now and stop opening yourself up to interaction, because strangers are always going to think worse of you for something. Hell, I think anyone who whistles or hums needs to be run up a flagpole by the underwear and shot at with fire ants because they are terrible people. Replace "anyone who whistles or hums" with "Touhou fans" and "needs to be run up etc." with "are pedo" and you see what I mean.
Somewhere out there, a lot of people don't like what you're doing - eating meat, creating carbon emissions, playing Touhou, going on 4chan and ED, wearing fur, driving American cars, driving Japanese cars, driving European cars, taking the bus or subway, etc.
Like what you like as long as it does no direct physical harm to yourself or others, and to hell with what the Internet thinks of your hobby.
Viva freaking Touhou, man, seriously. I enjoy this fandom and I enjoy interacting with other Touhou fans. I don't know half of what you're all talking about, but that's okay because I'm still learning, and it's fun being here and reading.
Viva Touhou. VIVA! VIVA!