TOPIC: Suggestion for future years: Beta testing

I was one of the GM's for room 3 Dancing stones, the workshop puzzle room where everything had to be returned to the proper location on a shelf.

Those of us who worked that dungeon probably know of the issues we had on day 1, where it was exceedingly difficult. Not only was there just way too much information in the room (Actually had one or two guests say "There's no way they expect us to read all of this is there?), but one of the items (Bag of silver pellets) wasn't mentioned anywhere in the papers, at least not in a way that would link it to the red bag. They also didn't have enough time to catch the hints that could have helped them in room 4 puzzle, as most successful parties solved it right at the horn, after one or two whammies.

By the start of the second day, we'd made a few tweaks (Underlined important information, put the silver pellets on the shelf to start), and we were looking at roughly a 50% pass rate for puzzle side, which while tough, was way better than the first day. That being said, the golden ticket runs, and the entirety of the Thursday guests were faced with a room that was not exactly fun (I had a lot of frustrated party members)

After the end of my Friday shift, I was discussing the end results with some of the backstage team members, and brought up the following idea.

When a new computer program comes out, there's a period of alpha and beta testing before it's released to the public. This is to make sure that it's not only "ready", but also helps eliminate errors that might have been missed by programmers. It's that whole thing where you need proof-reading, because although you're reading the right words in your mind, because it's what you expect, the actual code or words on the paper might be wrong and will only be noticed by someone who doesn't have previous knowledge.


What if part of the Wednesday setup was to take people (Possibly from the build crew) who have no knowledge whatsoever of the dungeon mechanics, and have a proof-read "alpha test" of each room? We'd be able to check for inconsistencies, anything that the designers may have missed. For the puzzle rooms, take a group of 10, and see if it's realistic for it to be solved in 12 minutes. For combat, check the slide boards for issues (Had one with an extremely slick center, but sticky gutters) and for room layout (Is there realistically enough room for 10 people + GM to maneuver for quick sliding).

Now, I'm aware that this is kind of the purpose of the golden ticket run. That being said, there's only about 8 hours between the end of the GT and the start of the next day, people have to sleep, and any last minute changes may not be as well thought out as if we'd had a few hours between the test run, and the start of the GT. This would truly make the GT a "Beta test", and might give the GT run a chance at having more polished fun experience, and give a chance to evaluate any changes before things get busy on Thursday.

Thoughts? I'm aware that there's already a LOT of work that goes into this, so this may not be as viable, but I do believe it would make a better experience for guests, especially as expensive as tickets have been getting. If players are paying 60+ per ticket, they're expecting something amazing, and I feel this would give it to them.

Adam Pillari wrote: I was one of the GM's for room 3 Dancing stones, the workshop puzzle room where everything had to be returned to the proper location on a shelf.

Those of us who worked that dungeon probably know of the issues we had on day 1, where it was exceedingly difficult. Not only was there just way too much information in the room (Actually had one or two guests say "There's no way they expect us to read all of this is there?), but one of the items (Bag of silver pellets) wasn't mentioned anywhere in the papers, at least not in a way that would link it to the red bag. They also didn't have enough time to catch the hints that could have helped them in room 4 puzzle, as most successful parties solved it right at the horn, after one or two whammies.

By the start of the second day, we'd made a few tweaks (Underlined important information, put the silver pellets on the shelf to start), and we were looking at roughly a 50% pass rate for puzzle side, which while tough, was way better than the first day. That being said, the golden ticket runs, and the entirety of the Thursday guests were faced with a room that was not exactly fun (I had a lot of frustrated party members)

After the end of my Friday shift, I was discussing the end results with some of the backstage team members, and brought up the following idea.

When a new computer program comes out, there's a period of alpha and beta testing before it's released to the public. This is to make sure that it's not only "ready", but also helps eliminate errors that might have been missed by programmers. It's that whole thing where you need proof-reading, because although you're reading the right words in your mind, because it's what you expect, the actual code or words on the paper might be wrong and will only be noticed by someone who doesn't have previous knowledge.


What if part of the Wednesday setup was to take people (Possibly from the build crew) who have no knowledge whatsoever of the dungeon mechanics, and have a proof-read "alpha test" of each room? We'd be able to check for inconsistencies, anything that the designers may have missed. For the puzzle rooms, take a group of 10, and see if it's realistic for it to be solved in 12 minutes. For combat, check the slide boards for issues (Had one with an extremely slick center, but sticky gutters) and for room layout (Is there realistically enough room for 10 people + GM to maneuver for quick sliding).

Now, I'm aware that this is kind of the purpose of the golden ticket run. That being said, there's only about 8 hours between the end of the GT and the start of the next day, people have to sleep, and any last minute changes may not be as well thought out as if we'd had a few hours between the test run, and the start of the GT. This would truly make the GT a "Beta test", and might give the GT run a chance at having more polished fun experience, and give a chance to evaluate any changes before things get busy on Thursday.

Thoughts? I'm aware that there's already a LOT of work that goes into this, so this may not be as viable, but I do believe it would make a better experience for guests, especially as expensive as tickets have been getting. If players are paying 60+ per ticket, they're expecting something amazing, and I feel this would give it to them.

+1. Good ideas here

Spoilers.
I was on the build, so I can say this fairly confidently. Last year we had a lot of issues and were working down to the wire and this would have been difficult. This year however we had a couple dozen people with little to do a few hours before the GT run. Part of the DM training could have been try a run of alpha volunteers that try to see/do

And even having done the DAS puzzle run three or four times, I never caught that the blower/furnace had instructions in the apothecary. doh. I never heard anyone else relate this before now.

I'm pretty sure the DAS puzzle room had a clue for the skrael mother/queen final battle for combat side. So that was less helpful that way too.

I'm totally for an alpha test, though it should be opt-in so people can avoid spoilers.

Yeah, it would have to be opt-in, if only to keep said testers from being exposed.


And yes, the room 3 puzzle did have information for both the blacksmith, and for the queen scrael fight, but again, I think of the three shifts I did in that room, I had only two parties finish with any kind of time left over. Most of the ones that did solve it did so after failing once and taking puzzle damage, and then finally getting the solution right as the horns were sounding. Definitely not enough time to pick up the hints for future rooms.

My groups on the whole loved the Dungeons, but I did have several complain from party members about how much reading was required in low-light rooms. The scent puzzles and the bowl / music puzzles both seemed to have technical problems as well, the bowls often made no noise (which caused us to fail), and I don't think the scents all matched up. For instance, pine cone scent should be easily discernable, but none smelled like pine cones to any of us.

Not sure if there would be time and resources for this, but it's worth considering. I think the puzzles get play tested before leaving Carbondale.

And I have to say that I was one of the ACs for Dancing and I didn't know there were any clues to later rooms in room 3.

Mike Steele wrote: My groups on the whole loved the Dungeons, but I did have several complain from party members about how much reading was required in low-light rooms. The scent puzzles and the bowl / music puzzles both seemed to have technical problems as well, the bowls often made no noise (which caused us to fail), and I don't think the scents all matched up. For instance, pine cone scent should be easily discernable, but none smelled like pine cones to any of us.

Same issue with bowl / music puzzle not making sound, room with only a single light source resulted in only 2 people working the puzzle and 8 more or less standing by. Scents were an issue as most also smell like plastic. The puzzle where you had to differentiate colors including white and yellow while only given yellow lights for the run.

bpsymington wrote: Not sure if there would be time and resources for this, but it's worth considering. I think the puzzles get play tested before leaving Carbondale.

As build, stuff gets left behind, stuff gets set up incorrectly. There was some confusion about the Tehlu's path puzzle, and part of it was cut up to fix it. It was found that it was right to begin with and was put back, but this just shows that more eyes are needed to see if a puzzle can be solved the right way, the wrong way, or can't be solved. Still might not catch everything, but it's better than no on-site testing.

I know one of the room 3 pages showed a lot of symbols but recognized the planetary symbols like Earth, Venus, Mars, Jupiter, Saturn, etc. I didn't see a page of the blower/furnace runes. But I easily could have missed it. It would have both made sense (that the apothecary had some notes relating to runes) and been an interesting clue for the later puzzle. But I can't confirm or deny. Now that it's torn down, only the TD people can confirm or deny it.

edwin wrote:

Mike Steele wrote: My groups on the whole loved the Dungeons, but I did have several complain from party members about how much reading was required in low-light rooms. The scent puzzles and the bowl / music puzzles both seemed to have technical problems as well, the bowls often made no noise (which caused us to fail), and I don't think the scents all matched up. For instance, pine cone scent should be easily discernable, but none smelled like pine cones to any of us.

Same issue with bowl / music puzzle not making sound, room with only a single light source resulted in only 2 people working the puzzle and 8 more or less standing by. Scents were an issue as most also smell like plastic. The puzzle where you had to differentiate colors including white and yellow while only given yellow lights for the run.

I wanted to mention this but forgot:

If colors of non-lighted things matter in the dungeon all the lights need to be the white ones.

The yellow ones make everything a monochrome, it's basically impossible to tell colors apart.

Lodestone (KH) wrote: I'm pretty sure the DAS puzzle room had a clue for the skrael mother/queen final battle for combat side. So that was less helpful that way too.

Yep, my party found that one. Fat lot of good that did us.

Of course, we didn't find the hint that was actually relevant, and naturally failed miserably on the blower puzzle.

Matthew Hayward wrote:

edwin wrote:

Mike Steele wrote: My groups on the whole loved the Dungeons, but I did have several complain from party members about how much reading was required in low-light rooms. The scent puzzles and the bowl / music puzzles both seemed to have technical problems as well, the bowls often made no noise (which caused us to fail), and I don't think the scents all matched up. For instance, pine cone scent should be easily discernable, but none smelled like pine cones to any of us.

Same issue with bowl / music puzzle not making sound, room with only a single light source resulted in only 2 people working the puzzle and 8 more or less standing by. Scents were an issue as most also smell like plastic. The puzzle where you had to differentiate colors including white and yellow while only given yellow lights for the run.

I wanted to mention this but forgot:

If colors of non-lighted things matter in the dungeon all the lights need to be the white ones.

The yellow ones make everything a monochrome, it's basically impossible to tell colors apart.

At the riddle reveal Jeff said they ordered all white lights but still had a bunch of yellow left over. Sounds like they will be phased out as they stop working though.