Irreal

Michał Poczwardowski has a nice post on Goodhart’s Law. The law is usually expressed as “When a measure becomes a target, it ceases to be a good measure.” That may seem a little abstract but its application is familiar to all of us.

My favorite example is rewarding schools or teachers for how well their students do on standardized tests. It seems like an obvious win: reward the schools and teachers who produce successful students. The problem is that that characterization is missing the part at the end that says, “as measured by standardized tests”. Of course, what actually happened is that teachers—either on their own or under direction from the school administration—started to “teach to the test”. Students didn’t actually learn more or become better students, they just became better at taking the tests. The reason I say this is my favorite example is because long ago I thought it was a great idea. Reward the good teachers and maybe even cull the bad ones. Sadly, it didn’t work out that way. Only the producers of standardized tests liked the results.

Another example is the infamous cobra effect, which, while it may be apocryphal, perfectly captures the idea. A seemingly reasonable measure for a desired result is incentivised and people find a way to maximize the measure instead of the desired goal. Very often, as with the Cobra effect, this makes the original goal worse.

Just as Goodhart says, as soon as the measure becomes the goal, people pursue it whether or not it actually helps achieve the desired result. Poczwardowski has some other modern day examples, including his own attempt to encourage engineer recruitment.

I know I’ve written about this before but it’s a lesson we forget at our peril. Take a look at Poczwardowski’s post. It’s a short and interesting read.

Someone reposted Trey Harris’ famous story about the 500 mile email. The TL;DR is that Harris, working as a university system administrator, received a call from the Statistics Department claiming that they couldn’t send an email further than about 500 miles. If you know anything at all about how email works, your reaction would be the same as Harris’: Yeah right.

Oddly, though, it turned out to be true. It was the Statistics Department, after all, and they had all sorts of data supporting the claim. Harris ran his own tests and discovered to his astonishment that it was true. Email to a site closer than 500 miles worked fine. Those to sites further away than a little over 500 miles invariably failed.

I’ve written about this at least a couple of times before but it’s such a good story that it’s worth repeating now and then for people who haven’t heard it. The solution makes perfect sense once you know what it is but until you do it’s seems like an impossibility.

Take a look at Harris’ original post for the answer and for the amusing story. You may even learn a new way of thinking about network problems. Sometimes you need to look under the layer where the problems seems to exist to discover what’s actually going on.

I was going to make the fact that Microsoft 365 and other Microsoft tools are down into a Red Meat Friday post with a title that included the word “schadenfreude” but my better nature prevailed and I forebore. Today, though, I saw this: they’re still down. To be fair, it appears to involve more than just Microsoft so it’s probably a network issue of some sort.

Nevertheless, this inconvenient fact remains: if a critical part of your company’s workflow depends on an Internet connection to operate you really should reconsider your decisions. Sometimes, an Internet connection is unavoidable but it certainly shouldn’t be for basic operations like word processing. Who benefits from the need for an Internet connection? Hint: it’s not the user.

It doesn’t have to be this way. There are plenty of open source office suites that are free to use and don’t require calling home to work. There are many companies and even governments that are using them today.

The whole thing reminds me of the old adage that “No one ever got fired for choosing IBM”. It was never true, of course, but it did express the widely held sentiment that IBM was the safe bet. The same doubtlessly holds for Microsoft but it’s even less true for them. Their software in no more robust—often less robust, in fact—than their competitors’ offerings that aren’t phoning home all the time to check your subscription or make sure that what you’re writing meets their terms of service as Google Docs famously does.

The general rule is easy to state. Anything critical should operate—and most importantly, store data—locally. Sure, some apps will need a connection to complete their task—WordPress for blogging is an example—but they shouldn’t need that connection just to start using the application.

Of course, I’ve been preaching this for years and people still aren’t listening. I don’t expect that to change. To be clear, I don’t insist that you have to use open source software, only that if the Internet goes down or the company goes out of business, you can still get your work done.