Packing C Structures

I’m writing this on my MacBook Pro, a small, light-weight laptop that has 16 GB of memory. Nothing I do on it ever runs out of memory. It wasn’t always that way, of course. Bill Gates famously quipped that he couldn’t imagine why anyone would ever need more than 64 KB of ram¹. We’re a long way from MS-DOS, of course, but my last laptop had only 2 GB of ram. It seemed like a lot when I got it.

In the old days, even mainframes had limited memory and people writing in C learned how to layout their data structures so that they would use minimal memory. It’s not something that I’ve worried about for some time but once it was second nature to think about such things.

Occasionally, the need to worry about memory comes up even today. Most often it’s a problem for embedded systems programmers but even programs on “normal” machines that deal with huge data sets can have problems. Eric Raymond, esr, ran into that with his cvs-fast-export program that he wrote to help with his conversion of CVS repositories to Git. When dealing with large repositories, he would routinely run out of memory. By a careful rearranging of his structure definitions, he was able to reduce his memory footprint by 40%

Sadly, knowledge of these techniques is dying out but sometimes they are still necessary. Happily, esr has written a document that explains the methods. They aren’t that hard and if you are a C programmer you should take a look at esr’s writeup and bookmark it for the time that you need it. If you’re writing in something like Python or Ruby, you won’t need the techniques directly but you may still find them useful for furthering your understanding of what’s happening under the covers.