Latency

Every programmer understands, at least in a general way, the relative latency of various computer operations. For example, we all know that accessing a value from a CPU register is faster than accessing that value from an L1 cache is faster than accessing that value from an L2 cache is faster than accessing that value from an L3 cache is faster than accessing that value from memory is faster than accessing that value from a disk.

We might even have an idea of what those speeds are. Typical values might be .9 ns for accessing an L1 cache versus 120 ns for a main memory access. If you’re like me, it’s hard to compare those values. They seem essentially the same: really small.

Here’s a wonderful spreadsheet that makes those comparisons come alive¹ by normalizing the values to a one second cycle time. With that, an L1 cache access takes 3 seconds and a memory access 6 minutes. No need to do mental calculations here; we can see immediately how much faster the L1 cache access is. To continue the comparison, a disk access takes 1–10 ms; normalized to the 1 sec cycle time that’s 1–12 months. See what I mean about making the comparisons come alive?

The spreadsheet considers actions such as sending an IP packet from NY to San Francisco or to the United Kingdom and booting the system. Take a look at the spreadsheet. The values are truly amazing. As a teaser, booting the system takes 32 millennia.

Of course, normalizing the times to a one second cycle time doesn’t really give us more information, it just makes the information more understandable. Looking at the values you begin to see why things like locality of reference really do matter.