BetaComp 2004 Results

After much delay, the results of this year’s BetaComp are in. I was pleased to see that once again, each entrant found at least *something* that no other entrant did, and no single entrant found more than about 50% of all the bugs that were found. This reinforces the prevailing theory that several testers (and possibly multiple rounds of testing) are most effective in ferreting out all the bugs. —Jess Knoch

—BetaComp 2004 Results (rec.games.int-fiction)

The “BetaComp” is a competition for beta-testing interactive fiction computer games. In the world of literature, a good editor with a sharp pencil can make the difference between a mediocre novel and a critically acclaimed classic.

Beta-testers play a game when the author thinks it’s finished. They often try solutions the author never dreamed of — some of which cause errors that make the game unwinnable, others of which are just annoyances. It’s up to the author to use the beta report(s) to revise the game, either so that the alternative solutions the beta-testers tried all work, or there is some belilevable (or at least humorous) in-game reason why a legitimate attempt is disallowed.

The winner is Graham Holden, who found a lot of bugs and wrote a detailed report for the benefit of the designer.

I found it much more effective to ask one or two beta-testers to look at my work in progress, then as soon as I get the reports, fix the problems and send it off to another few testers.

The other option, which is sending the same version out to six or eight people at once, means that many of the testers will be frustrated by the same bugs, and you’ll have scores of pages of transcripts and suggestions to go through.