While working on my own forthcoming book The Inside Game (due out April 21st from HarperCollins; pre-order now!), I stumbled across a chapter from Prof. Gary Smith’s book Standard Deviations: Flawed Assumptions, Tortured Data, and Other Ways to Lie with Statistics, a really wonderful book on how people, well-meaning or malicious, use and misuse stats to make their arguments. It’s a very clear and straightforward book that assumes no prior statistical background on the part of the reader, and keeps things moving with entertaining examples and good summaries of Smith’s points on the many ways you can twist numbers to say what you want them to say.
Much of Smith’s ire within the book is aimed at outright charlatans of all stripes who know full well that they’re misleading people. The very first example in Standard Deviations describes the media frenzy over Paul the Octopus, a mollusk that supposedly kept picking the winners of World Cup games in 2010. It was, to use the technical term for it, the dumbest fucking thing imaginable. Of course this eight-legged cephalopod wasn’t actually predicting anything; octopi are great escape artists, but Paul was just picking symbols he recognized, and the media who covered those ‘predictions’ were more worthy of the “fake news” tag now applied to any media the President doesn’t like. Smith uses Paul to make larger points about selection bias and survivorship bias, about how some stories become news and some don’t, how the publish-or-perish mentality at American universities virtually guarantees that some junk studies (found via p-hacking or other dubious methods) will slip through the research cracks, and so on. This is more than just an academic problem, however: One bad study that can’t survive other researchers’ attempts to replicate the results can still lead to significant media attention and even steer changes in policy.
Standard Deviations bounces around a lot of areas of statistical shenanigans, covering some familiar ground (the Monty Hall problem and the Boy or Girl problem*) and less familiar as well. He goes after the misuse of graphs in popular publications, particularly the issue of Y-axis manipulation (where the Y axis starts well above 0, making small changes across the X-axis look larger), and the “Texas sharpshooter” problem where people see patterns in random clusters and argue backwards into meaning. He goes after the hot hand fallacy, which I touched on in Smart Baseball and will discuss again from a different angle in The Inside Game. He explains why the claims that people nearing death will themselves to live through birthdays or holidays don’t hold up under scrutiny. (One of my favorite anecdotes is the study of deaths before/after Passover that identified subjects because their names sounded “probably Jewish.”) Smith’s reach extends beyond academia; one chapter looks at how Long-Term Capital Management failed, including how the people leading the firm deluded themselves into thinking they had figured out a way to beat the market, and then conned supposedly smart investors into playing along.
* Smith also explains why Leonard Mlodinow’s explanation in Drunkard’s Walk, which I read right after this book, of a related question where you know one Girl’s name is Florida is incorrect, and thank goodness because for the life of me I couldn’t believe what Mlodinow wrote.
I exchanged emails with Smith in September to ask about the hot hand fallacy and a claim in 2018 by two mathematicians that they’d debunked the original Amos Tversky paper from 1986; he answered with more detail that I ended up using in a sidebar in The Inside Game. That did not directly color my writeup of Standard Deviationshere, but my decision to reach out to him in the first place stems from my regard for Smith’s book. It’s on my list now of books I recommend to folks who want to read more about innumeracy and statistical abuse, in the same vein as Dave Levitan’s Not a Scientist.
Next up: About halfway through Mary Robinette Kowal’s The Calculating Stars.
is a beautiful polemic straight from the headquarters of the Statistical Abuse Department. Seife, whose 
, a somewhat dated look at the rise of Big Data in decision-making that has since been lapped by the very topic it attempted to cover.) If done correctly, the margin of error should equal two standard deviations, but many journalists and pundits treat it as some ambiguous measure of the confidence in the reported means. When Smith is leading Jones 51% to 49% with a margin of error of ±3%, that’s not a “statistical dead heat;” that’s telling you that the poll, if run properly, says there’s a 95% chance that Smith’s actual support is between 48% and 54% and a 95% that Jones’ support is between 46% and 52%, with each distribution centered on the means (51% and 49%) that were the actual results of the poll. That’s far from a dead heat, as long as the poll itself didn’t suffer from any systemic bias, as in the famous Literary Digest poll for the 1936 Presidential election.
.