Pun Salad

I enjoyed reading Daniel Kahneman's classic, Thinking Fast and Slow (belatedly, back in 2017). And even though that work might have relied too much on research of doubtful replicability, I still was tempted to read this latest work. It's co-authored with Olivier Sibony (apparently a business consultant type), and Harvard lawprof and popular non-fiction author Cass Sunstein.

The book discusses and summarizes a specific way "judgments" go wrong. Specifically, when humans are presented with the same set of relevant facts about a situation, and are asked to decide on a specific conclusion, they "should" come up with the same answer. But they don't.

The authors distinguish two kinds of judgment flaws: bias, where the decision process is giving consistently wrong answers; and noise, where the answers are scattered widely. They propose a simple experiment: take out your smartphone, pull up the clock, which probably has a lap timer function. Use that to (for example) try to measure 6 (or so) "laps" of 10 seconds each, without looking. (For extra credit, do it without counting in your head.)

Your average lap time will (almost certainly) not be exactly 10.0 seconds; this is bias: your inner clock is running fast or slow. But your results will also (again, almost certainly) scatter around that average, and that's noise.

Noise in judgment is very often bad. Examples used in the book: judges vary widely in the sentences they impose on criminals guilty of the same offense, with similar histories and situations. That can be due to the judges having lenient/strict sentencing standards, but "studies show" it can also be due to the time of day, whether the judge's home team lost its last game, what they had for lunch, … That's not the way we'd like to think the justice system works.

Other examples are drawn from the business world: setting insurance premiums, making hiring decisions, doing performance evaluations, the merger and acquisition process, etc. Here, excessive noise in judgment can result in dysfunction and corporate ruin. Unsurprisingly, there have been a lot of studies done on the sources of noise, although noise hasn't risen to the popularity of its partner in crime, bias.

The book describes a number of strategies to minimize noise, mostly in the corporate/government spheres. For example, they encourage more reliance on noise-free "algorithms" to substitute for flawed human judgments.

I would have liked to see a little more emphasis on how individuals can tame their inner noisiness, but the diligent reader can probably construct some useful personal advice from the book's discussion.

To their credit, the authors consider criticisms of noise-reduction. For example, they look at Cathy O'Neil's book Weapons of Math Destruction, which purports to show how "algorithms are increasingly used in ways that reinforce preexisting inequality."

(Note: I suspect that often means: "algorithms give us answers we don't like.")

And they often consider cases where a certain amount of noise can be beneficial. For example, it can cause your decision-making to evolve and adapt to changing environments.

So: it's an interesting read, a little dry in spots, and my interest waned in the business-intensive sections. (I could imagine that corporate execs could find those extremely interesting, though.)