Michał Poczwardowski has a nice post on Goodhart’s Law. The law is usually expressed as “When a measure becomes a target, it ceases to be a good measure.” That may seem a little abstract but its application is familiar to all of us.
My favorite example is rewarding schools or teachers for how well their students do on standardized tests. It seems like an obvious win: reward the schools and teachers who produce successful students. The problem is that that characterization is missing the part at the end that says, “as measured by standardized tests”. Of course, what actually happened is that teachers—either on their own or under direction from the school administration—started to “teach to the test”. Students didn’t actually learn more or become better students, they just became better at taking the tests. The reason I say this is my favorite example is because long ago I thought it was a great idea. Reward the good teachers and maybe even cull the bad ones. Sadly, it didn’t work out that way. Only the producers of standardized tests liked the results.
Another example is the infamous cobra effect, which, while it may be apocryphal, perfectly captures the idea. A seemingly reasonable measure for a desired result is incentivised and people find a way to maximize the measure instead of the desired goal. Very often, as with the Cobra effect, this makes the original goal worse.
Just as Goodhart says, as soon as the measure becomes the goal, people pursue it whether or not it actually helps achieve the desired result. Poczwardowski has some other modern day examples, including his own attempt to encourage engineer recruitment.
I know I’ve written about this before but it’s a lesson we forget at our peril. Take a look at Poczwardowski’s post. It’s a short and interesting read.