A scout I’ve seen a few times already this spring on the amateur trail recommended Jerry Muller’s brief polemic The Tyranny of Metrics, a quick and enlightening read on how the business world’s obsession with measuring everything creates misaligned incentives in arenas as disparate as health care, education, foreign aid, and the military, and can lead to undesirable or even counterproductive outcomes. With the recent MLB study headed by physicist Prof. Alan Nathan that found, among other things, that players trying to optimize their launch angles hasn’t contributed to rising home run rates, the book is even somewhat applicable to baseball – although I think professional sports, especially our favorite pastime, do offer a good contrast to fields where the focus on metrics leads people to measure and reward the wrong things.
The encroachment of metrics on education is probably the best known of the examples that Muller provides in the book, which is strident in tone but measured (pun intended) in the way he supports his arguments. Any reader who has children in grade school now is familiar with the heavy use of standardized testing to measure student progress, which is then in turn used to grade teacher performance and track outcomes by schools as well, which can alter funding decisions or even lead to school takeovers and closings. Of course, I think it’s common knowledge at this point that grading teachers on the test performance of their students leads teachers to “teach to the test,” eschewing regular material, which may be important but more abstract, in favor of the specific material and question types to be found on these tests. My daughter is in a charter school in Delaware, and loses more than a week of schooldays each year to these statewide tests, which, as far as I can tell, are the primary way the state tracks charter school performance – even though charters nationwide are rife with fraud and probably require more direct observation and evaluation. That would be expensive and subjective, however, so the tests become a weak proxy for the ostensible goal in measurement, allowing the state to point and say that these charters are doing their jobs because the student test scores are above the given threshold.
Muller mentions sports in passing in The Tyranny of Metrics but focuses on other, more important industries to society and the economy as a whole; that’s probably a wise choice, as the increased use of metrics in sports is less apt than the other examples he chooses in his book. However, there are some areas where his premise holds true, with launch angle a good one to choose because it’s been in the news lately. Hitters at all levels are now working with coaches, both with teams and private coaches, to optimize their swings to maximize their power output. For a select few hitters, it has helped, unlocking latent power they couldn’t get to because their swings were too flat; for others, it may help reduce flyouts and popups and get some of those balls the hitter already puts in the air to fall in for hits or go over the fence. But for many hitters, this emphasis on launch angle hasn’t produced results, and there are even players in this year’s draft class who’ve hurt themselves by focusing on launch angle – knowing that teams measure it and grade players in the draft class on it – to the exclusion of other areas of their game, like just plain hitting. Mike Siani of William Penn Charter has cost himself a little money this spring for this exact reason; working with a coach this offseason to improve his launch angle, he’s performed worse for scouts this spring, becoming more pull-conscious and trying to hit for power he doesn’t naturally possess. He’s a plus runner who can field, but more of an all-fields hitter who would benefit from just putting the ball in play and letting his speed boost him on the bases. Because many teams now weigh such Trackman data as launch angle, spin rate, and extension heavily in their draft process, either boosting players who score well in those areas or excluding those who don’t, we now see coaches trying to ‘teach to the test,’ and that approach will help only a portion of the draft class while actively harming the prospects of many others.
At barely 220 pages, The Tyranny of Metrics feels like a pamphlet version of what could easily be a heavy 500-page academic tome, recounting all of the ways in which the obsession with metrics produces less than ideal results while also explaining the behavioral economics principles that underlie such behavior. If you have some of that background, or just don’t want it (understandable), then Muller’s book is perfect – a concise argument that should lead policymakers and business leaders to at least reconsider their reliance on the specific metrics they’ve chosen to measure employee performance. Using metrics may be the right strategy, but be sure they measure what you want to measure, and that they’re not skewing behavior as a result.
Next up: I’m currently reading Ray Bradbury’s short story collection I Sing the Body Electric!.
Interesting. The last company I worked at learned this…. As with most companies, agent call time is a big deal. Agents were incented to get off the phone fast. That rarely was good for the customer. Somehow we did an experiment where we had some get rated on other factors, and it turns out that being on the phone longer and helping customers actually pays off….
When you create a metric to measure performance, people will perform to the metric.