The Iron Law of Data Collection Strikes the UK

I’ve written before about the Iron Law of Data Collection, the idea that no matter the rationale given for the collection of the data or the promises made about its confidentiality there will always be mission creep that finds new uses for the data and its eventual abuse. The two links above give examples of it in action. Now, we have another example, this time from the UK.

In a sense, this is the worst betrayal of those I’ve written about. The government promises you that they have to collect your financial information in order to administer the income tax system but that that information is held in absolute confidence and will not be released outside the government1. That promise is necessary to prevent large scale cheating and resistance to the HMRC.

The UK case is particularly troubling because the reason for breaking the promise is so venal. The HMRC is proposing the sell the information to “third parties,” including private companies. As one of the commenters said, “…it has proven there is still life in the 19th century adage ‘the political classes would sell their own grandmothers for a shilling’.

Sadly, the Iron Law marches on. Here’s the next example of its application.



In the US, at least, the release of tax payer information is a serious felony. I assume the same is true in the UK. Of course, even in the US there’s been recent abuse for political reasons. One more reason, if you needed it, not to trust their high sounding promises.

Posted in General | Tagged | Leave a comment


An excellent suggestion from Ryan Paul (via Jean-Philippe Paradis)

Posted in General | Tagged | Leave a comment

Encrypt Everything?

Klint Finley over at Wired has an article that discusses the pros and cons of encrypting the Internet. By that he means using SSL/TLS instead of plain HTTP for every connection. It’s not a new idea. For example, the EFF has its HTTPS Everywhere program that includes a plugin for many browsers that always tries to establish an HTTPS connection even if the site defaults to HTTP.

I really like this idea, at least in principle. It makes man-in-the-middle attacks difficult if not impossible and goes a long way towards keeping the NSA’s nose out of our business. On the other hand, does it really make sense for, say, me to force HTTPS connections to Irreal? There’s certainly no first order reason to do so. There’s no sensitive data being transferred and Irreal is hardly subversive enough that anyone would worry about having the government or anyone else see what they’re reading on the site.

Of course there’s the benefit that it makes the Internet a sea of unintelligible bits and thus makes government snooping really difficult. Perhaps that’s reason enough. What do you think? Should all sites—even the innocuous like Irreal—use SSL/TLS or is the benefit not worth the cost?

Posted in General | Tagged | Leave a comment

An Emacs Lisp Based Common Lisp

Lars Brinkhoff has a really interesting project up at GitHub. It’s emacs-cl, a Common Lisp implemented in Emacs Lisp. This probably isn’t all that useful but it sure is awesome. As far as I can tell, it’s a pretty complete. CLOS and pretty printing are missing but it has lexical closures, packages, multiple values, bignums, adjustable arrays, and other CL features.

This is one of those things that is noteworthy not because it’s going to change your life or be particularly useful but because it shows what a tool you already have can do. It’s often said that Emacs Lisp is crippled and not powerful enough to do real work. Guys like Nic Ferrier and Kris Jenkins have already put the lie to that notion but here is yet another example of what Emacs Lisp can do.

I’ve said several times that Emacs is the closest thing we have to a Lisp machine but mostly we don’t pursue that aspect of Emacs. Brinkhoff’s project reminds us that Emacs really does provide a Lisp environment and that the Lisp it provides is far from toothless.

Posted in Programming | Tagged , | 4 Comments

Emacs 24.4 Pretest

The first pretest of Emacs 24.4, Emacs 24.3.90, is available via FTP at the GNU Emacs archive. This the first of an expected 6 or so pretest releases leading up to Emacs 24.4.

The Emacs devs are now in bug quashing mode so if you don’t mind life on the bleeding edge, download the pretest and help find the remaining bugs.

Posted in General | Tagged | 4 Comments

Handling Password Fields

With the advent of the heartbleed debacle you’ve probably spent a bunch of time changing your passwords. I know I have. Having to update several passwords has opened an old wound: the really really stupid policies and coding behind password fields.

My passwords are generated automatically by my password manager. They’re long random strings containing both cases, symbols, and digits. They’re very secure. Of course a lot of sites don’t like that. They’re too long, or they have symbols the site doesn’t like, or some other fool thing. I saw all these complaints while I was changing my passwords.

Here’s Terence Eden with a hillarious post on entering passwords. He’s got all the problems (and some that are, maybe, just a bit fanciful). Except for the obvious humor, they’re all real and all very annoying. But the worst, absolute worst, is the common practice of not telling you what the password policy is. You try a password and get an error message. You try another and get another error message. It’s like a text adventure game. I know, how about xyzzy1? Eden’s post captures this craziness nicely.

A good general rule, which I subscribe to, is that if there are any restrictions on the password then the site is insecure. That’s because if they’re doing it right—using bcrypt, scrypt, PBKDF2 or something similar—then it doesn’t matter what password you enter2. If you’re not using bcrypt or one of its siblings then it’s almost a certainty that you’re not properly hashing and key stretching so that when your user database is captured it will be an easy matter to recover the passwords.

Eden suggests some rules to reduce user frustration. The first, of course, is state your policy up front so the user doesn’t have to guess. Of course, if you’re doing it right you don’t need any rules because you won’t have any restrictions. Yes, yes that means that users can input stupid passwords but most stupid passwords will pass the normal rules anyway. Password123 anyone?



I tried that one but nothing happened.


OK, if you’re too lazy to deal with variable length inputs, choose a max password size of 1024 or something that no one is apt to actually use. They all get hashed down to the same size anyway so the initial size doesn’t matter. The point is that arbitrary restrictions are a sign of weakness.

Posted in General | Tagged | 4 Comments

Sacha Chats with Tom Marble

Sacha Chua has posted another interesting episode in her series of video chats with Emacs users. This time it’s with Tom Marble who talks about and demonstrates his Emacs-based invoicing system. Marble is a consultant and wrote the system to track the details and hours of his consulting work and to generate nice looking invoices for billing.

Marble also talks about using Emacs for his Clojure work and he demonstrates some of his work flow. As usual, Chua has him take us through a quick trip of his .emacs file. The video is just over 57 minutes so plan accordingly.

If you like the video and want to see more, Chua has a page with links to all her Emacs chats. I invariably learn some tidbit of Emacs goodness that I didn’t know so the videos are definitely worth your time.

Posted in General | Tagged | Leave a comment

The NSA and Heartbleed

Bloomberg is reporting that the NSA knew about and exploited the heartbleed bug for years. The NSA is, of course, denying that but if you’ve been keeping score it’s pretty easy to figure out who to believe.

If it’s true that the NSA knew about the bug and kept the information private so that they could use it, it’s a nice illustration of something that Bruce Schneier has said repeatedly. The NSA is making us less safe not more safe. It’s virtually impossible to know if a site has been exploited by heartbleed so it’s entirely possible, or even likely, that criminals or foreign state actors have been exploiting the bug. The NSA’s primary mission is protection of the country’s communications. If they failed to act to close a devastating hole in a widely used SSL implementation then they’re obviously failing in that mission. Failing on purpose.

Posted in General | Tagged | Leave a comment

Mickey on Running Shell Commands with Dired

The essential Mickey over at Mastering Emacs has another post in his series on dired. This time it’s about running shell commands on multiple files from within Emacs. If you’re a Unix user and used to using find and xargs to do this sort of thing, you should definitely check out Mickey’s post. Using dired is easier and you don’t have to leave Emacs or even bring up a shell of some sort in a buffer. Unless you’re a dired expert, Mickey’s post will teach you things you didn’t know. And while you’re there, be sure to follow the link to his post on how to work with files spread out across many directories. The two posts tell you pretty much everything you need to know to use dired in situations that would otherwise require find and xargs.

If you’re like me, you probably started using dired as a convenient way of listing files so you could pick one to load into a buffer. Later, maybe, you started doing simple operations like deleting a file or perhaps renaming it. It’s when you first realize that you can work on multiple files at the same time that you begin to realize how powerful dired is. For example, watch Magnar Sveen use multiple-cursors to change file names in a writable dired buffer. Of course you can do this using dired commands too but it shows how flexible dired is and how it can adapt to your way of working. Mickey’s post shows how you can apply external commands to multiple files in a variety of ways. Very useful information.

Posted in General | Tagged | Leave a comment


Every programmer understands, at least in a general way, the relative latency of various computer operations. For example, we all know that accessing a value from a CPU register is faster than accessing that value from an L1 cache is faster than accessing that value from an L2 cache is faster than accessing that value from an L3 cache is faster than accessing that value from memory is faster than accessing that value from a disk.

We might even have an idea of what those speeds are. Typical values might be .9 ns for accessing an L1 cache versus 120 ns for a main memory access. If you’re like me, it’s hard to compare those values. They seem essentially the same: really small.

Here’s a wonderful spreadsheet that makes those comparisons come alive1 by normalizing the values to a one second cycle time. With that, an L1 cache access takes 3 seconds and a memory access 6 minutes. No need to do mental calculations here; we can see immediately how much faster the L1 cache access is. To continue the comparison, a disk access takes 1–10 ms; normalized to the 1 sec cycle time that’s 1–12 months. See what I mean about making the comparisons come alive?

The spreadsheet considers actions such as sending an IP packet from NY to San Francisco or to the United Kingdom and booting the system. Take a look at the spreadsheet. The values are truly amazing. As a teaser, booting the system takes 32 millennia.

Of course, normalizing the times to a one second cycle time doesn’t really give us more information, it just makes the information more understandable. Looking at the values you begin to see why things like locality of reference really do matter.



I can’t figure out the provenance of the spreadsheet to give proper credit. If you know, leave a comment.

Posted in Programming | Tagged | Leave a comment