Author Topic: Touhou TAS identification experiement (Read 21616 times)

I have no name · « **on:** May 26, 2012, 08:13:02 AM »

I had an...interesting...idea for an experiment. As far as I can tell, most people will take a run that looks legit and pass it off as legit without a second thought. If it looks TASed, then it must be a TAS and the poster is a cheater. I'm sure this is an exaggeration, but that's not why I'm creating this thread.

I'm creating this thread to show how easy it is to create TASes that can look legit, and with enough tries, legit runs that look TASed. I used Imperishable Night's spell practice to create 15 replays, some of which are TASed and others are not. I will not say how many of each there are.

I'm attaching a .zip with the 15 replays inside it. If you would like to participate, all you have to do it watch the replays and PM to me which ones are TASed, and which ones are legit. After enough time has passed/enough people have responded, then I want to see how the 'community' does-discussion is encouraged in this phase.

Think of it as an optional true/false test.

Please do not post answers/guesses in the thread until I give the ok, feel free to post in the thread saying you sent in answers, feel free to discuss TASing in general for Touhou, and if other runs are/aren't TASes, etc. As long as you aren't posting your guesses here until I give the ok.

@mods: if this is not a topic that should be brought up/discussed then feel free to lock/delete the thread. It's only an idle curiosity.

Esper · « **Reply #1 on:** May 26, 2012, 03:20:24 PM »

Answers sent.

Ranko Hoshino · « **Reply #2 on:** May 26, 2012, 03:54:21 PM »

sent my answers

SuccinctAndPunchy · « **Reply #3 on:** May 26, 2012, 05:10:25 PM »

Sent mine too.

DarkDespair5 · « **Reply #4 on:** May 26, 2012, 05:45:33 PM »

I'm betting I have an error rate of >50% (lol) but here goes.

fsvgm777 · « **Reply #5 on:** May 26, 2012, 07:46:53 PM »

I've sent my answers as well.

Naut · « **Reply #6 on:** May 26, 2012, 09:12:14 PM »

It's really irrelevant which ones are TASed and which aren't. The whole point of using tools would be to do something well beyond your capability to brag to other people about (well, this is the only tool use worth calling out). These kinds of things would be considered more personal accomplishments, in which case you're only cheating yourself out of the satisfaction of doing them. Hiding tool use only really becomes an issue if the person was doing something that mattered to the majority of the community; eg. competing in high score threads or anything that involves direct competition with another person. If you're not competing with somebody else then it really doesn't matter if the replay was tool assisted or not. Play the games however you want.

In addition, tool assisted replays are usually on really difficult things that also span a long amount of time (eg. high score runs) so there are more places for the player to screw up hiding their tools (or demonstrate that those two or three amazing dodges were flukes). So for replays of such minimal length, it really doesn't matter if they're using tools or not since the accomplishment probably doesn't matter to many people and it's something you could probably fluke through if it's beyond your ability anyway (DS 9-7).

touhoumaniac · « **Reply #7 on:** May 26, 2012, 09:15:26 PM »

Looking forward to seeing the results.

Spoiler:

I have thought about this issue. I don't know how ZUN's games work, but i have made some bullethell games with replay ability from scratch.

SuccinctAndPunchy · « **Reply #8 on:** May 26, 2012, 09:42:46 PM »

Quote from: Naut on May 26, 2012, 09:12:14 PM

It's really irrelevant which ones are TASed and which aren't. The whole point of using tools would be to do something well beyond your capability to brag to other people about (well, this is the only tool use worth calling out). These kinds of things would be considered more personal accomplishments, in which case you're only cheating yourself out of the satisfaction of doing them. Hiding tool use only really becomes an issue if the person was doing something that mattered to the majority of the community; eg. competing in high score threads or anything that involves direct competition with another person. If you're not competing with somebody else then it really doesn't matter if the replay was tool assisted or not. Play the games however you want.

In addition, tool assisted replays are usually on really difficult things that also span a long amount of time (eg. high score runs) so there are more places for the player to screw up hiding their tools (or demonstrate that those two or three amazing dodges were flukes). So for replays of such minimal length, it really doesn't matter if they're using tools or not since the accomplishment probably doesn't matter to many people and it's something you could probably fluke through if it's beyond your ability anyway (DS 9-7).

I agree with this post wholeheartedly since it more or less echoes my initial thoughts on the experiment , but it's fun to try and spot the TAS, no? I see this as a test of my understanding of the systems. Which I'm probably going to bomb horribly but eh, is fun!

Quote from: Naut on May 26, 2012, 09:12:14 PM

probably fluke through if it's beyond your ability anyway (DS 9-7).

*looks at 1000+ shot 90FPS pre-clear*

*looks at 4 shot 90FPS pre-clear on different savefile*

OH GOD WHY, IT'S TRUE :qq:

BT · « **Reply #9 on:** May 26, 2012, 10:07:25 PM »

Sent them in anyway. I want to see how wrong I am. :U

Star King · « **Reply #10 on:** May 27, 2012, 12:23:04 AM »

Sent. I hope I don't fail miserably :X

Esper · « **Reply #11 on:** June 01, 2012, 05:05:19 PM »

Are you ever going to post the answers? :V

I have no name · « **Reply #12 on:** June 01, 2012, 06:35:31 PM »

Quote from: Ace on June 01, 2012, 05:05:19 PM

Are you ever going to post the answers? :V

Yes.
I was kind of hoping for at least one more person to give guesses though, but ah well.

ANSWERS BELOW

Spoiler:

1: The point was to make a TAS that looked real. I played this at 50% speed, but used big motions. The giveaway, if any was in the number of last second reactions. (5 correct guesses)
2: The point was to make a real run that looked TASed. This was played at 60 FPS with as much jittery movement as possible, and as many microtaps as I could safely fit. (5 correct guesses, everyone got these both right or both wrong)
3: (mostly) no focus First Pyramid. First try at that! This was a test to see if adding a challenge made it look TASed. (7 correct. Come on guys, i can NF with the best of them :V)
4: People can do this, why can't I? The giveaway is the focus/unfocus at the end being perfectly timed to the time point gauge, if any. Played at 20 FPS. (6 correct. There's a better method I tried to do legitimately, but couldn't within a reasonable amount of tries, so have this standalone)
5: obviously fake no focus TAS. Played at 30 FPS, put in to create confusion with the other 2. (6 correct? That means 3 wrong? This was supposed to be obviously fake...how did 2 people put this as real but the First Pyramid as TAS?)
6: legitimate no vertical Woo capture. Looks pretty TAS right? (4 correct. 5 people were fooled by this-I got a no vertical Woo capture

)
7: Played at 45 FPS. This one is meant to be borderline, especially after the last 2. He wouldn't put 3 TASes of the same card in a row, would he? The answer is no, I wouldn't, this time. (yeah uh...so much for borderline. 9 correct responses out of 9)
8: Legitimate run, involving a lot of messing around. Fooled a lot of you right? (5 correct. Funny thing about this one-it fooled 4 of the first 5 responses, but the last 4 all got it perfect -_-)
9: legit run. Microtapping isn't THAT hard. (5 right, 4 wrong again. This took about a dozen tries actually)
10:50% TAS, giveaway is the unfocusing on wave 3. (6 right. Nothing more to say)
11:legit run, designed to look as similar to the TASed run as possible. (6 right again, but only 4 people got both of these right. 1 got both wrong)
12:25% TAS. Obvious. The focusing at the beginning was done at 60 FPS though. This gives an obvTAS, making 'legit' have a fuzzier boundary... (LOLTAS but 3 people put this as real. 6 right answers)
13:50% TAS. Looks mostly legit due to the other TAS right before it. The dodge at the end has completion bias-of COURSE he got lucky, it's the successful run! (only 5 right answers- 7 people got one of the 2 Fantasy Heavens' wrong, and the other 2 got both right.)
14:This was the only purely obvious TAS I intended to create. 25% speed, giveaway is the unfocused dodges at the start. (yeah uh...I'm not that good, despite what 3 of you seem to think. 6 right)
15:legit run. I might try this in an actual run since it's not very hard. (7 right, 2 wrong. Said wrong answers were both from people who got the prior wrong)

I had 9 people give answers, so here's a stat dump.
Mean correct answers: 11.444444
Median correct answers: 8
Data set: 7 8 8 8 8 9 11 14 15
Perfect score: touhoumaniac
Almost perfect: BT (missed number 6)
C grade: Seppo Hovi (11 right)
"person who thought they did horribly but didn't": RankoHoshino (9)
"hey let's all get the same score": GamingAwesome, Esupanitix "Ace", DarkDespair5, Star King (8/15)
oops: fsvgm777 (7 correct)
score if I take the majority choice: 14/15, only failing number 6
replay correctly guessed most: number 7, the solo Alice Woo run. everyone got it right.
replay incorrectly guessed most: number 6, the no vertical Woo run. 4 people got it right, 5 people got it wrong.

Final summary: despite how simple it can be to make a TAS that doesn't look like a TAS, it can be called out if enough people look at it. Likewise, a legit run can also be wrongly called out as a TAS, although the majority seems to be right in most cases. The majority successfully pegged every TAS, but got one legitimate run wrong. This also shows there are multiple TAS giveaways-precision movement, insane reflexes, even staying under the boss perfectly can be one. If I want to know if something is a TAS, I'm taking the majority of touhoumaniac, BT and Seppo Hovi because their majority got everything right.

I guess now if anyone else wants to run something like this now is the time, I know DarkDespair5 expressed interest.

touhoumaniac · « **Reply #13 on:** June 01, 2012, 07:22:42 PM »

Quote from: I have no name on June 01, 2012, 06:35:31 PM

Perfect score: touhoumaniac

:toot:

I'm going to rewatch all the replays with the answer commentary.

Esper · « **Reply #14 on:** June 01, 2012, 07:30:22 PM »

Speaking of TAS discussion, how DO you use whatever you use for TAS runs? I'd like to use it just for unlocking stages.

Also fuck an 8/15 is 2.7/5 which is 5.4/10 which means I got an F

Could you post which ones I got right and which ones I got wrong, if you want? :>

I have no name · « **Reply #15 on:** June 01, 2012, 07:33:02 PM »

Quote from: Ace on June 01, 2012, 07:30:22 PM

Speaking of TAS discussion, how DO you use whatever you use for TAS runs? I'd like to use it just for unlocking stages.

Could you post which ones I got right and which ones I got wrong, if you want? :>

Cheat Engine, speedhack, set to a number less than 1.

You got 1, 2, 6, 9, 11, 12 and 14 wrong.
Come on, 12 and 14 were obviously TASes :V

Esper · « **Reply #16 on:** June 01, 2012, 07:49:12 PM »

Quote from: I have no name on June 01, 2012, 07:33:02 PM

Come on, 12 and 14 were obviously TASes :V

WELL TO BE FAIR ABOUT 14 YOU DID THE WOO NOVERTICAL CAPTURE LEGITIMATE SO WHAT DO YOU EXPECT

Also I can't turn on speedhack. DLL injection failed: Failed to allocate memory.

BT · « **Reply #17 on:** June 01, 2012, 07:57:56 PM »

Oh hey, I knew 4 TAS in a row (4, 5, 6, 7) was odd but so was that No-Vert Woo. :V

Ranko Hoshino · « **Reply #18 on:** June 01, 2012, 08:01:54 PM »

Eye got ⑨ right? Eye'm such a genius! :V

fsvgm777 · « **Reply #19 on:** June 01, 2012, 08:20:27 PM »

I'll be honest: I had some doubts on several of the replays. Oh well, even if I'm last, I did it for fun.

..at least I got 7 right.

SuccinctAndPunchy · « **Reply #20 on:** June 01, 2012, 09:08:52 PM »

8/15, huh?

This batting average pleases me.

Glad to have participated and not completely sucked ass at it.

No Vert Woo cap just to fuck with us kind of amused me, honestly. :3

Karisa · « **Reply #21 on:** June 02, 2012, 03:01:30 AM »

Quote from: Ace on June 01, 2012, 07:49:12 PM

Also I can't turn on speedhack. DLL injection failed: Failed to allocate memory.

Try the vsync patch then. Have you ever played at 90fps? The same method should work for slowing the game down, if you set it to 30 or 15 or something instead.

Esper · « **Reply #22 on:** June 02, 2012, 03:02:34 AM »

Quote from: Karisa on June 02, 2012, 03:01:30 AM

Try the vsync patch then. Have you ever played at 90fps? The same method should work for slowing the game down, if you set it to 30 or 15 or something instead.

It doesn't do that.

Karisa · « **Reply #23 on:** June 03, 2012, 04:34:40 AM »

That's strange... anything below 60 seems to act the same as 60. I obviously never tried to slow the game down with it myself, but I thought it would work. I wonder why it doesn't?

Drake · « **Reply #24 on:** June 03, 2012, 07:55:50 AM »

because the purpose of the vsync patch isn't to let you crap all over the game

SuccinctAndPunchy · « **Reply #25 on:** June 03, 2012, 09:52:57 AM »

Quote from: Drake on June 03, 2012, 07:55:50 AM

because the purpose of the vsync patch isn't to let you crap all over the game

Wee bit harsh, what's wrong with allowing the user more control over the game? The more functions, the better, right?

VSync patch leaves a mark on the FPS as well so it's not like you'd be able to get away with it.

Still, I can't tell you what to do with your own creation but that struck me as rather hostile.

Seppo Hovi · « **Reply #26 on:** June 03, 2012, 12:29:11 PM »

I still don't understand why should people find the Woo NV replay TAS. Seriously, there was nothing suspicious at all, the movements and the reading were both currently followable even by my standards. The major factor was luck.
For example, had this been TAS, IHNN would've been making a lot of short-reaction time dodges and such at this point instead of standing still and hoping for a good luck.

I have to also note, that quite a few of the TAS replays here were purely idiotic, since they didn't use any scoring tactic or do a trick that would increase one's score magically. They were mainly survival TAS's, which are rather silly to watch in my opinion.

Also, when looking out a TAS replay, the first thing you look at is how new the player is, if they have any replays available, and if there is any suspicious stuff about the replay the player submits. For example, if a completely new person you've never heard of before submits something really awesome, you probably should check out if it has any signs of TAS. For example, I had to look at both Random's IN extra Marisa and IS replays for a while, until I could admit that they were legit. He had fast as fuck reaction time, and screwing up Phoenix Tail (but capturing rings, this is rather odd to be honest) while having those reflexes and being comfortable with the focus-tapping was another thing I was surprised of. Also, if there is no replay submitted in the first place people might get angry or suspicious. Or at least I might.

The main thing when you look for a TAS is the movement. If there is movement that clearly outdoes human muscles (e.g. some of the dodging IHNN does during one of his Fantasy Heaven replays here has tapping too fast for normal humans) , it is a sure sign of TAS. We're all bound to humanity in that way, and there is no running away from that.

Another good thing to look at is if you can follow up to the bullet reading speed of the player. It has only happened to me once with a legit run, and that was by an inhuman monster on his own grounds. And I don't consider myself that bad when it comes to following a player's movement. However, TAS replays often tend to read everything too fast for everyone, since it's TAS'd and has several extra seconds of reaction speed for the player. Of course it might be luck and just randomly swinging around, but I really doubt that one would get that lucky several times during a full run.

Random · « **Reply #27 on:** June 03, 2012, 05:14:25 PM »

Quote from: Seppo Hovi on June 03, 2012, 12:29:11 PM

Also, when looking out a TAS replay, the first thing you look at is how new the player is, if they have any replays available, and if there is any suspicious stuff about the replay the player submits. For example, if a completely new person you've never heard of before submits something really awesome, you probably should check out if it has any signs of TAS. For example, I had to look at both Random's IN extra Marisa and IS replays for a while, until I could admit that they were legit. He had fast as fuck reaction time, and screwing up Phoenix Tail (but capturing rings, this is rather odd to be honest) while having those reflexes and being comfortable with the focus-tapping was another thing I was surprised of. Also, if there is no replay submitted in the first place people might get angry or suspicious. Or at least I might.

...I won't blame you for that. Watching that replay again, I would have probably said it was a TAS as well...

Glowing bullets have always been my worst enemy. That and jellybeans.

touhoumaniac · « **Reply #28 on:** June 03, 2012, 07:50:09 PM »

How to detect save states? :3

Drake · « **Reply #29 on:** June 03, 2012, 10:05:10 PM »

Quote from: GamingAwesome1 on June 03, 2012, 09:52:57 AM

Wee bit harsh, what's wrong with allowing the user more control over the game? The more functions, the better, right?

VSync patch leaves a mark on the FPS as well so it's not like you'd be able to get away with it.

Still, I can't tell you what to do with your own creation but that struck me as rather hostile.

You misunderstand. I didn't make the Vsync patch; I packaged it up, added english instructions and I manage the download mirror. I say it isn't the purpose of the Vsync patch because literally, it is not the purpose of the program. The reason why it wouldn't go under 60fps is because naturally, Touhou plays at 60fps and is drawn on a 60Hz monitor. The patch takes over drawing and can adjust to help vsync issues as well as input lag. It can also adjust the FPS to match the monitor's refresh rate and a bunch of other tiddly possibly useful things. It was never meant to be used to speed up or slow down the game, it was meant to make the game play as normal as possible. Altering the FPS is just a side-effect that users can take advantage of.