Jakob Nielsen’s latest column, about the dangers of quantitative studies, is out. In a nutshell, he argues that numbers lie. Misrepresentation of statistical significance, confusing correlation and causality, ignoring covariant variables, over-simplifying analysis, and out and out distortion of the measurements are of course all potential pitfalls in any research, and it’s good to point them out. Of course, I have a copy of How to Lie with Statistics on my bookshelf that made the same point fifty years ago—and made it more entertainingly.
There’s also an interesting connection to Mark Hurst’s column from two weeks ago about the “Page Paradigm.” Nielsen, who has been advocating breadcrumb navigation and other usability features for years, rises like a brook trout to Hurst’s bait that breadcrumbs, unless they help the user fulfill their one purpose in visiting, are useless. He makes cogent points about users who drop into a site from a search engine or other outside links, or who might want to revisit a site, but still I have to wonder whether the timing of that particular discussion is coincidence.
Still, I have to agree with his fundamental point that quant isn’t everything. I was recently in a situation where we had mounds of hard quantitative data—we were, almost literally, drowning in it—but couldn’t solve our fundamental challenge. Had we been able to run a few focus groups, we could have zeroed in on the problem much more effectively.