Why Usability Testing Matters --
Palm Beach County Ballot Design Raises Questions about Election
08 Nov 2000;
Dennis G. Jerz
The result of the 2000 U.S. Presidential race was so close that some Democratic Party officials argue that one Florida county's hard-to-use ballot may have unfairly decided the presidency.
Critics argue that some voters in Palm Beach County, Fla. might have accidentally voted for Reform Party candidate Pat Buchanan, when they thought they were voting for Al Gore. The Democrats are listed second in the left column; but punching a hole in the second circle actually cast a vote for Buchanan.
Image of the Palm Beach County election ballot (first published in the Orlando Sun-Sentinel)
The Reform Party is the second highest ticket listed on the ballot; hence, it seems logical that Buchanan's name should be connected to the second hole. But some voters may have never noticed Buchanan's name, if all they intended to do was find the hole for Gore, punch it, and go home. (Historical footnote: Buchanan almost wasn't on the Florida ballot at all!)
The ballot does in fact feature arrows leading from the candidate's name to the appropriate holes -- but the arrows are short, making the visual connection line weak. Assuming that a voter knows what the arrows are for, the form makes perfect sense; nevertheless, anyone who has ever conducted a usability test knows that people are generally bad at figuring out all but the simplest documents (even when they have access to instructions).
Usability testing involves asking a small test population -- sometimes no more than five people -- to use a prototype, and noting any problems. Since people are rarely willing to spend time learning how to use each of the dozens of documents they encounter each day, technical writers and designers strive to create documents that do not mislead or confuse.
While the designer of the Palm Beach County ballot presumably knows why the names are presented in this particular order, the order is not clear to the casual reader. Since the layout of this ballot was not clear, some people may have had to guess.
The Associated Press quoted voter Eileen Klasfeld as saying, "It was so hard to tell who and what you were voting for. I couldn't figure it out, and I have a doctorate."
Boca Raton resident Blake Smith told an AP reporter that he had to ask for another ballot: "When I went to push the one for president, I pushed one and it seemed to be just below the office of vice president. It seemed like I had to push one for vice president, too. Then I saw I had accidentally voted twice."
Another 19,000 ballots in Palm Beach were thrown out because people seem to have voted twice; this problem is potentially far more significant, given the tightness of the race. Republicans will doubtless try to determine whether a similar number of ballots were spoiled in previous elections.
Questions began to surface after Buchanan received 3,407 votes in this heavily Democratic district -- far more votes than he (a conservative former Republican) had received in some far less liberal districts. Since Florida's initial tally showed Bush winning Florida by only about 1,700 votes, it makes sense for the Democrats to investigate this anomaly.
Florida election officials from both parties have called the ballot straightforward, and Republicans argue that claims of voter confusion exaggerated. My guess is that if the Florida ballot design remains an issue, a lot of political pundits and lawyers will get crash courses in document design.
08 Nov, 2000 -- first posted
09 Nov, 2000 -- minor edits, update
15 Nov, 2000 -- added more links
26 Sep, 2001 -- added another link
(Update: 9 Nov, 2000)
By now, Florida is undoubtedly brimming with lawyers. Nobody ever notices technical writing or document design unless something goes wrong. There are probably just as many poorly-designed ballots that led people to vote for Gore by mistake, which means that if the design of one ballot is challenged in court, thousands of ballots across the nation will likely be under scrutiny. All of this only emphasizes LePore's mistake in not user-testing the ballot in the first place.
(Update: 26 Sep, 2001)
Pregnant Chad Can't Vote Anymore (Wired)
The home of the butterfly ballot and the pregnant chad announced Tuesday that it's spending $14 million to purchase some of the most sophisticated voting machines in the country.
(Update: 12 Nov, 2001)
Florida Recounts Would Have Favored Bush (Washington Post)
Under every scenario used in the study, the winning margin remains less than 500 votes out of almost 6 million cast.
(Update 23 Sep, 2002)
A Vote for Touch and Go Away [Florida Ballots] (Wired)
"Florida found itself in another voting debacle during last week's primary election: Tens of millions of dollars spent to abolish hanging chads and they still can't get it right."
19 Jul 2000; by Dennis
Usability Testing: What Is It?
The first rule of technical writing is "know your audience." But even the best planning cannot predict all possible user errors. This document introduces the concept of testing for usability, which measures whether test subjects can actually use your prototype to complete assigned tasks.
19 Jul 2000; by Dennis
Prototypes in Technical Writing: What Are They?
A good prototype will help you identify flaws (research gaps or mistaken assumptions) long before you have dug yourself into a hole by investing a lot of time in it. A sculptor makes a scale model in clay -- a prototype -- before chiseling away at a full-sized chunk of marble. It is much easier to fix major mistakes in clay than it is to throw away a ruined chunk of marble and start over again.
Another Example of Ballot Designer's Technical Communication Skills:
(In the same ballot, LePore was re-elected to her $107,000-a-year job as county elections supervisor.)
Terry LePore Ballot Designer Fan Page