TOPIC: Gathering Class Damage Data! Fabulous(?) Prizes!

Endgame wrote:

David Harris wrote: I like the idea. Just curious is there an anticipation that the live game data would differ significantly from just testing different builds within the app or website. Specifically, I have all 10 builds on my phone and could run multiple rounds of simulated combat against various ACs and record the results of hits and damage. Theoretically, you could run many/many rounds of simulated combat at any time.

Simulated data won’t covering things like - you missed 3 attacks because you attacked the wrong mirror image, or you missed your will save and were affected by insanity or went for a swim, or missed your fort save and died in the first combat round.

I hear your concern, but you can simulate any of the conditions you mentioned above. Live data assumes you know every action the party members are taking. I will sometimes consume a potion of bull's strength and add it to my calculation, but see no need to tell the DM or party. They just need to know my end result. And while I'm not aware of anyone who's ever done this, what's to keep someone from a miss, but telling the DM they hit. As a wizard, i've thrown some fairy dust or dream dust damage onto a spell, but don't think it's critical to announce it. I just add it to my reported damage and throw the token into the "Jeff made me give this to you" pile. Then there are just honest mistakes. You misread the monster AC based on difficulty and report a hit when it should have been a miss (or vice versa). You also need to make sure you know the damage types of each party member. If any of my damage type is X, the app will just automatically increase the damage done to any monster whose weakness is X. What if the bard is incapacitated and bard song gets turned off after round 1, but someone forgets to turn it off. So many, many mistakes can creep into the data.

What is the hypothesis that we are collecting data on? You have alluded to what it is not, but not what you are interested in testing. Sharing that could help this community develop a solid test method and data recording.

My concern is that with the spreadsheet as is, any conclusions about relative damage rates between classes will not work well. This is because it treats all classes as having roughly equivalent builds under the selected party heading. If any player is outfitted at a different level (higher or lower) compared to the party, that class's data will be skewed and we will have no way of knowing that. The sheet might work at a party level for comparisons of total DMG (ex: UC-R vs UR+), but i am not sure.

Endgame wrote:

David Harris wrote: I like the idea. Just curious is there an anticipation that the live game data would differ significantly from just testing different builds within the app or website. Specifically, I have all 10 builds on my phone and could run multiple rounds of simulated combat against various ACs and record the results of hits and damage. Theoretically, you could run many/many rounds of simulated combat at any time.

Simulated data won’t covering things like - you missed 3 attacks because you attacked the wrong mirror image, or you missed your will save and were affected by insanity or went for a swim, or missed your fort save and died in the first combat round.

Isn't that the kind of thing you want to exclude when looking for apples to apples damage calculations. Warcraft DSP numbers for example used to be based off a program called MEW that would run thousands of repeated combat simulations under identical circumstances. Even if the circumstances were wrong so long as the mechanics were modelled correctly you could at least speak to how everyone behaved under the same wrong circumstances so you could isolate the effective DSP chance caused by say adding 50 more hit.

Not to poopoo the idea but its seems like getting a dozen different people to record data from a dozen runs with luck based mechanics like mirror images, or variable solution rooms like the subdual/regular/remove curse will introduce tones of errors and inconsistency into the data.

dave romanin wrote: What is the hypothesis that we are collecting data on?

This probably is the most important question to understand before a method of gathering data can best be determined.

Picc wrote:

Endgame wrote:

David Harris wrote: I like the idea. Just curious is there an anticipation that the live game data would differ significantly from just testing different builds within the app or website. Specifically, I have all 10 builds on my phone and could run multiple rounds of simulated combat against various ACs and record the results of hits and damage. Theoretically, you could run many/many rounds of simulated combat at any time.

Simulated data won’t covering things like - you missed 3 attacks because you attacked the wrong mirror image, or you missed your will save and were affected by insanity or went for a swim, or missed your fort save and died in the first combat round.

Isn't that the kind of thing you want to exclude when looking for apples to apples damage calculations. Warcraft DSP numbers for example used to be based off a program called MEW that would run thousands of repeated combat simulations under identical circumstances. Even if the circumstances were wrong so long as the mechanics were modelled correctly you could at least speak to how everyone behaved under the same wrong circumstances so you could isolate the effective DSP chance caused by say adding 50 more hit.

Not to poopoo the idea but its seems like getting a dozen different people to record data from a dozen runs with luck based mechanics like mirror images, or variable solution rooms like the subdual/regular/remove curse will introduce tones of errors and inconsistency into the data.

If you want to see “true damage” during an adventure, these are important things to take into account.

For v4b nightmare (spoilers)
Room 1: 2/3 of single target attacks miss. Scroll fireball, Multi target wizard spells, and other AoE attacks will deal damage normal.

Room 3
3 monsters with Retribution damage pushes the party to ranged attacks. Melee based combatants are more likely to miss with ranged. Ranged combatants (ex ranged Ranger, spells) are unaffected. Scroll fireball, Multi target wizard spells, and other AoE attacks will deal extra damage.

Room 5. Failing insanity check or other saves during this combat incur a hit penalty Harsh enough that you will likely miss (-10 hit IIRC). Auto hit attacks, like scrolls and non sliding wizard spells are unaffected as far as I know.

Room 6 subdual
Damage must be melee based to subdue. There may be benefits to different damage types (does blunt subdue better than slashing?). Melee characters will do best here.

Room 7:
Fort save or be petrified will have a huge effect here if you fail your save early. AoE damage is unaffected.

In other words, a real world test will have dramatically different results than a simulated data set that does not take these conditions into effect. Since we don’t play simulated TD, those numbers are less valuable than the real world numbers.

I 100% support gathering more data. I though think Fiddy is very correct.

Though VTD sounds like it may continue in some capacity when we get back to physical TD, I also would caution that data for VTD is not the same as data for physical TD for combat. Also the player base of VTD is VERY different than the player base of physical TD so conclusions drawn should keep that also in mind.

I agree that real world data will more accurately represent the damage being reported during a real dungeon. But what is it you want to do with the data? It might be very interesting, but I'm not sure what you're trying to prove. I certainly wouldn't base build, class or token design decisions based on the data because of the enormous number of variables which could skew the data one way or another. In the end, you'll have a huge discussion/debate around the data and only a few people agreeing upon the outcome.

And then the biggest variable comes into play later this year. A real dungeon, where skill of sliding comes back into play, will look nothing like VTD.

Dave wrote: I agree that real world data will more accurately represent the damage being reported during a real dungeon. But what is it you want to do with the data? It might be very interesting, but I'm not sure what you're trying to prove. I certainly wouldn't base build, class or token design decisions based on the data because of the enormous number of variables which could skew the data one way or another. In the end, you'll have a huge discussion/debate around the data and only a few people agreeing upon the outcome.

And then the biggest variable comes into play later this year. A real dungeon, where skill of sliding comes back into play, will look nothing like VTD.

So ... what should build, class, or token design decisions be based on?

Lots of people claiming their way of playing is the one, true way? Supported by made up builds that use tokens the poster doesn't have and that aren't what someone else would use? Supported by real builds that are suited for a particular group and a particular collection not shared by other people?

Gathering more data is not any worse than what presently exists. Sure, it's going to have tons of problems with it. Maybe it always will. Maybe someone will be able to create a better iteration. I like to track how many rounds combat lasts on my runs, but I don't share that data because it's not relevant to the plurality of runs, but it gives me 2% clearer idea what my play is like. Maybe sharing data doesn't end up having any meaning, but it might interest people who are gathering it to see actual numbers on their own runs.

Dave wrote:

dave romanin wrote: What is the hypothesis that we are collecting data on?

This probably is the most important question to understand before a method of gathering data can best be determined.

Considering the data is being collected via self-reporting, knowing the hypothesis he's looking to test sounds like it would be a great way to create biased data.

Endgame wrote: Room 6 subdual
Damage must be melee based to subdue. There may be benefits to different damage types (does blunt subdue better than slashing?). Melee characters will do best here.

Yes, blunt damage works better than slashing.

I heartily agree this approach isn’t perfect.

There is no specific hypothesis I am testing, and it’s often the case that data collected absent any particular hypothesis is valuable to later researchers or theorists so that does not concern me.

I appreciate the ideas for improvement posted here - but with one day to go I’m trying to follow the KISS principle and not let the perfect be the enemy of the good.

If we get some good collaboration, and some data that seems generally useful to the community here, that can form the basis for a more structured, detailed scope for future dungeons.

Finally - with regard to simulation data - we’ve had nothing but simulation data for years and discussions about class damage do not seem to reach any sort of consensus. I hope true dungeon (ba-dum-bum-tiss) data will help inform those discussions in a way that simulation data can’t.

OK, i'm game to help. Do you want total DMG for multiple hits or keep each hit's DMG separate?