Author Topic: DNHAPM: Profiling and Game Monitoring for Touhou Danmakufu ph3 [20180723: v1.0]  (Read 9592 times)

Sparen

  • Danmakufu Artist
  • Git ready, git set, PUUSH!
    • AFCDTech
Hello Danmakufu ph3 scripters!

Over the past few weeks I've been tinkering around with Danmakufu a lot more than usual, experimenting with shaders and render targets for my RaNGE 19 Contest Entry, building data structures and implementing bit operators (sort of) in Danmakufu, and more recently, utilizing File Objects.

So today I'd like to announce a new project - DNHAPM, a profiling and game monitoring tool/system/script for Touhou Danmakufu ph3.

What is DNHAPM?
DNHAPM is a Danmakufu ph3 script that can be run in any stage/plural/single (recommended that you use a stage). It monitors playthroughs of your script, logs data, and on request, will spit out a report.

What do I mean by report? Well, take a look below!

FPS Monitoring


Look at those lag spikes! At the S# notes (spell card declarations), the FPS tanks due to all of the fancy primitive effects in my spell card cutin animation. And the lag spike at 3m20s is probably the boss explosion - enough particle effects to knock any computer below 60 FPS!

But this FPS profiler also catches some less savory issues - gradual FPS drops near the ends of Nonspells 2 and 3. Maybe I should check those out and see if there are objects that aren't being deleted, hmm...

Graze Monitoring


70 graze in the first nonspell?! Wow! The graze graph shows which spell cards the player grazed in. Interesting for level design, that's for sure!

Here you can see where heavy grazing occurred, or where grazing was particularly low.

Score Monitoring


Here you can see where your players are getting their score from (assuming score is enabled). This will probably correlate strongly with the graze graph, though with spikes for spell card captures, bullet clear bonuses, and other bonuses, like the stage clear bonus at the end.

Enemy Bullet Count Monitoring


Another interesting graph comes our way. With the enemy bullet count graph, you can see every time the player bombs (to an extent), as well as now the bullet count fluctuates over time.

In spells 2 and 3, the peaks are all around the same height - that's really good, since it means that all the bullets are deleting properly. But maybe having 1200 bullets on screen at once is a little problematic? Hmm... the graph still beats dredging through the lag-inducing log window.

Death & Bomb Monitoring


This graph shows where the player died and bombed (image does not show bombs or prior runs). Taking a look at the graph gives a good sense of which parts are the deadliest. Some attacks are easier than others~

User-Defined Monitoring
No graph for this one, unfortunately, as this depends on what you need. Want to log points? Power? Your custom game-specific feature? If someone's cheating during an all-horizontal script? This is the tool for you :)

Usage
So what is this, really? It's a tool - for scripters - that takes data from the game and displays it in HTML + CSS + SVG for you to analyze. It's highly customizable, and more features are coming soon!

To use the project, head over to the GitHub page and go to Releases. The SparenDNHAPM_#.#.tar.gz file contains the project README as well as the .dnh file. The README on GitHub gives the full rundown of the project + configuration and usage and other details, but to summarize here:

- Run the SparenDNHAPM.dnh script via @Loading in your target script
- Ensure that you've configured everything the way you like
- Ensure that you're outputting the report at a good time (DNHAPM requires at least 4 frames of buffer time between calling the event for report generation and closing the stage scene)
- Add annotations in your spells/wherever you want to add notes
- Run! voila!

There are some quirks that are detailed in the README, especially in regards to aggregating multiple runs, but this tool (still in development!) should be helpful to some scripters.

Potential use cases:
- Embed into your release, distribute to beta testers, have them return the .csv files. Run the script on aggregate mode and look for trends in the graph (i.e. everyone is lagging here; something went wrong here; etc). Analyze, fix bugs, and profit.
- Log runs to look for specific ways to optimize/find pain points in your script. For example, there is a bomb & death profiler that logs player deaths and bombs - should be interesting to see where players tend to die.
- ... and many more.

LATEST VERSION: 1.0 [2018 JUL 23]

Have feedback, comments, bug reports, suggestions, etc? Please feel free to post here or create an issue on GitHub. This project is in continuous development and knowing what best helps other scripters make balanced, optimized games will help me tailor the project/add features that benefit the Danmakufu community.

--Sparen of Iria
« Last Edit: July 23, 2018, 02:41:43 PM by Sparen »

Drake

  • *
Re: DNHAPM: Profiling and Game Monitoring for Touhou Danmakufu ph3
« Reply #1 on: June 29, 2018, 10:25:45 PM »
this is ultra cool

A Colorful Calculating Creative and Cuddly Crafty Callipygous Clever Commander
- original art by Aiけん | ウサホリ -

Gregory

  • I draw stuffs
Re: DNHAPM: Profiling and Game Monitoring for Touhou Danmakufu ph3
« Reply #2 on: June 30, 2018, 12:33:02 AM »
This is extremely neattttttttt

like, the best thing to check your bullets and fps dropping before putting scripts to public

Sparen

  • Danmakufu Artist
  • Git ready, git set, PUUSH!
    • AFCDTech
Re: DNHAPM: Profiling and Game Monitoring for Touhou Danmakufu ph3
« Reply #3 on: June 30, 2018, 01:06:45 AM »
this is ultra cool

This is extremely neattttttttt

like, the best thing to check your bullets and fps dropping before putting scripts to public

Thanks! But since this project is only a few days old, I haven't actually actively developed using the tool yet, so I currently don't know how well it functions in a proper testing environment (with Pausing, Continues, Packages, etc).

Helepolis

  • Charisma!
  • *
  • O-ojousama!?
Re: DNHAPM: Profiling and Game Monitoring for Touhou Danmakufu ph3
« Reply #4 on: June 30, 2018, 06:53:17 AM »
I am linking this in the Great Information Thread.

Sparen

  • Danmakufu Artist
  • Git ready, git set, PUUSH!
    • AFCDTech
Re: DNHAPM: Profiling and Game Monitoring for Touhou Danmakufu ph3
« Reply #5 on: July 01, 2018, 03:14:48 AM »
Version 0.3 has been released

What's new in 0.3:
- Bomb and Death Monitoring
- Clarifications on buffer time required post-event call to ensure that the report is written before the script is terminated

I wanted to provide an example of the Bomb & Death monitoring system but I realized literally at the end of a run of DNH Troll Stars that I had forgotten to add the event to actually write the files and generate the report, so no images for now.

Re: DNHAPM: Profiling and Game Monitoring for Touhou Danmakufu ph3
« Reply #6 on: July 02, 2018, 05:04:08 PM »
this is an incredibly useful tool for bullet hell scripters, mind you.. :3 helps with resource hogging and game balance type issues that are notorious for being time-consuming in game development
check http://vsoft.life/ for runs & creations
latest stream: Hollow Song of Birds Hard 1cc (OrangeReimu) @ https://youtu.be/eDFd4FHEjXg

Sparen

  • Danmakufu Artist
  • Git ready, git set, PUUSH!
    • AFCDTech
Re: DNHAPM: Profiling and Game Monitoring for Touhou Danmakufu ph3
« Reply #7 on: July 02, 2018, 06:38:19 PM »
Version 0.4 has been released.

What's new in 0.4:
- Step Logging and colored graph overlays + Spellcard event listening
- Bullet and Text size configuration settings for Death & Bomb Monitoring

Download the new release here: https://github.com/Sparen/Sparen-DNH-APM/releases/tag/v0.4

Please report all bugs on GitHub Issues. Please keep in mind that this project is a work in progress - although it probably works fine for smaller projects, some additional things need to be worked out before it can be used conveniently in a full game or a system with significant player or difficulty based changes.

Sparen

  • Danmakufu Artist
  • Git ready, git set, PUUSH!
    • AFCDTech
Version 0.5 has been released.

What's new in 0.5:
- User-defined Logging
- Custom file paths
- Reading only runs of the specified script during aggregation

Download the new release here: https://github.com/Sparen/Sparen-DNH-APM/releases/tag/v0.5

Please report all bugs on GitHub Issues.

This is expected to be the last version before 1.0, and contains the needed adjustments for the APM to work conveniently in a full game environment as well as in more complex Single scripts with multiple players and difficulties.

As usual, I welcome feature requests and comments on how to improve the APM, as it is a tool for the Danmakufu community to use.

Drake

  • *
I've sent a pull request, here are some additions:

I changed the FPS monitor to track FPS based on time passed rather than using DNH's function. This means you have much greater control over the tick. I haven't tested it a whole lot but this is how I tend to do it myself. One caveat is that for a reason I can't quite pin down this method consistently reports slightly less FPS than GetCurrentFps (likely somehow because it's all calculated from script rather than by the program itself), so there's a manual constant factor to fix it to be more or less consistent with what GetCurrentFps reports on average.

I've added a line to mark 60FPS on the graph as this will generally be what people want to target. I've also set 60 FPS to be the minimum value of the top of the graph for the same reason (if your program is somehow always running under 60FPS).

I've changed the line drawings so it moves the beginning of the graph to the first coordinate rather than at the static (0, 0) spot, mainly because having the 0 FPS at the beginning of every graph was annoying me and it's technically incorrect. The other things start at 0 so it didn't affect those, but it would affect FPS along with other user-defined stuff.

Here are pictures of 10-frame update and 2-frame update:



There's also an oversight affecting at least the FPS graph where the previously aggregated graphs all stay exactly where they were previously drawn even if the y-axis scale is changed due to there being a higher maximum value. This is made very clearly apparent if you play a normal run and then frameskip a bit on the next. This basically means the aggregates can't be compared (i.e. the whole point) if the maximum value changes.
« Last Edit: July 06, 2018, 10:15:58 AM by Drake »

A Colorful Calculating Creative and Cuddly Crafty Callipygous Clever Commander
- original art by Aiけん | ウサホリ -

Sparen

  • Danmakufu Artist
  • Git ready, git set, PUUSH!
    • AFCDTech
Pull request has been merged.

There's also an oversight affecting at least the FPS graph where the previously aggregated graphs all stay exactly where they were previously drawn even if the y-axis scale is changed due to there being a higher maximum value. This is made very clearly apparent if you play a normal run and then frameskip a bit on the next. This basically means the aggregates can't be compared (i.e. the whole point) if the maximum value changes.

I can reproduce the issue and will look into it. Confirming that it affects all graphs besides death/bomb.

EDIT: This issue has been fixed. After more testing, I will release a 0.6 with the contents of the pull request as well as fixes for this issue as well as any other issues I find.

EDIT 2: I've added a field that allows users to choose between your (Drake) method and GetCurrentFps, with the former being the default.
« Last Edit: July 06, 2018, 06:13:40 PM by Sparen »

Sparen

  • Danmakufu Artist
  • Git ready, git set, PUUSH!
    • AFCDTech
Version 0.6 has been released.

Changes in 0.6:
- New FPS monitoring option, thanks to Drake
- Fixes to Aggregation settings involving an oversight in regards to how max values were calculated (bug report thanks to Drake)

Download the new release here: https://github.com/Sparen/Sparen-DNH-APM/releases/tag/v0.6

Please report all bugs on GitHub Issues.

While this version should be stable (hopefully) and ready for use in short-term projects, I advise against integrating DNHAPM midway into a long-term project until 1.0 has been released, unless you want to risk having to adjust things later on.

As usual, I welcome feature requests and comments on how to improve the APM, as it is a tool for the Danmakufu community to use.

Sparen

  • Danmakufu Artist
  • Git ready, git set, PUUSH!
    • AFCDTech
Version 1.0 has now been released, after some testing on EUB and realizing that there were some system issues that needed to be fixed.

You can find the release here: https://github.com/Sparen/Sparen-DNH-APM/releases/tag/v1.0

Changes from 0.6:
- Script now automatically terminates after report generation instead of continuing to listen to events, potentially making Danmakufu run out of memory
- Correction factor on FPS logging is now customizable

If there are any issues, comments, or feature requests, please post here or, preferably, on GitHub issues.