It’s a dangerous mistake to believe that statistical research is somehow more scientific or credible than insight-based observational research. In fact, most statistical research is less credible than qualitative studies. Design research is not like medical science: ethnography is its closest analogy in traditional fields of science. —Jakob Nielsen —Risks of Quantitative Studies (Alertbox)
Note… he’s specifically talking about research into design — he’s not saying qualititaive research is always better than quantitative research.
Similar:
Flannery O'Connor reading "A Good Man is Hard to Find"
It's striking to hear her audience erupt...
Aesthetics
On this date in 1999 I first added a date to the list of web links I had been curating. (T...
On this date in 1999, I first dated an e...
Cyberculture
Umberto Eco and his legacy in open-world games
I read "The Name of the Rose" during the...
Books
The Unstoppable Rise of the Digital Content Creator
This is why I have been encouraging blog...
Business
Shatner's live, extemporaneous post-touchdown monologue on mortality was better than Kirk'...
After returning to Earth in Jeff Bez...
Awesome
Daughter Carolyn plays Cherry in Prime Stage's production of The Outsiders (Mar 6-15)
My daughter Carolyn plays Cherry in Prim...
Books
Confirmation bias: http://skepdic.com/confirmbias.html
I just wrote a blog entry about the “10% of people are gay” meme that makes a similar point. Given the option of choosing between two sets of data, we tend to choose the set of data that supports our beliefs, and discount the evidence against it. Scientists and academics and journalists are, of course, trained to counter this, but we aren’t always critically alert for signs of bias.
A trend found from multiple studies that were all poorly formulated and conducted doesn’t increase the credibility of the findings. Focusing on statistics from poor studies are no worse than any other conclusion from poor studies.
Six years ago, researchers showed that many of the most influential studies into usability had serious flaws, and that less-influential studies with different results did not have such flaws. The fact that the flawed studies are still the most influential today does not make the findings any more valid.