Literate Benchmarking

I’ve written several posts that celebrate Org mode for its ability to document various tasks in a literate way, execute the code, and gather the results. The nice thing about this process is that you have everything together:

  • Documentation of the task;
  • Code to perform the task;
  • Results from running the code.

They’re all in a single file that makes it easy to share and reproduce.

My favorite example of this is Howard Abrams’ post and video on Literate DevOps. His presentation is specialized to DevOps but the techniques he uses are easily adaptable to other types of tasks.

Alex Bennée has a nice example of using these techniques for running benchmarks. The problems with running benchmarks are similar to those encountered in DevOps so the same sort of “literate” solution can be applied. The post doesn’t show the entire benchmarking file but Bennée does provide a pointer to it. The file is on GitHub and therefore gets reformatted so if you want to see what’s really happening, click on the Raw button to see the actual Org file.

If you’ve been a developer for a while, your tendency—or at least mine—is to just run the necessary commands from the command line and be done with it. That’s quick and easy but now you have to remember exactly what you did to share the results or to repeat the task at a later date. I’ve been making a concerted effort to slow down and do these tasks in a literate fashion using Org mode. The extra time and effort pay off in the end and you can even link to the file from your lab notebook—you are keeping one, right?—so that everything is documented.

This entry was posted in General and tagged , . Bookmark the permalink.