Lisp At JPL

A long time ago, I read and enjoyed Ron Garret’s story of using Lisp at the Jet Propulsion Laboratory. Now, he retells the story in a Google Talk that is well worth watching. Part of the story explains how they were able to debug a problem when the DS1 spacecraft was a 10 or 45 (it’s not clear from the video) light minutes round trip away because the Lisp image that was running the program had a REPL.

Another story involved the Galileo magnetometer that was controlled by a Forth program. One of the bytes on the computer running Forth went bad so the engineers had to modify the program to avoid using the bad byte. The problem was that the program was originally developed using an Apple II Forth environment and the Apple II systems had long since been decommissioned. They solved this problem by recreating the entire Apple II Forth environment in Lisp.

If you enjoy hearing Lisp stories or stories about space flight and exploration then you should invest 70 minutes and watch the video.

Posted in Programming | Tagged | Leave a comment

Clozure Common Lisp 1.8

I took advantage of the weekend to upgrade my machines to Clozure CL 1.8. As usual, installation was a snap. Just get the distribution with SVN:

svn co http://svn.clozure.com/publicsvn/openmcl/release/1.8/darwinx86/ccl

for Mac OS X and then

ccl64 -n
? (ccl:rebuild-ccl :full t)

and you’re done. Users of other platforms will need to change darwinx86 to the directory appropriate for their systems as explained on the Getting Clozure CL page.

Lisp programs compiled with CCL 1.8 run a bit faster than those compiled with CCL 1.7 and, of course, there are various bug fixes and other clean up as described in the release notes.

I’ve only started using it but so far I’m happy with what I see. There are no obvious incompatibilities except for a couple of improvements to the directory function, and a minor change to the way the kernel handles a single command line argument that makes it easier to call standalone binaries. With performance enhancements and bug fixes, there’s not much to dislike.

Posted in Programming | Tagged | Leave a comment

Elisp Challenge No. 2

The other day, I found a reference to the Add-A-Gram puzzle, which is one of the retired challenges from ITA Software for prospective employees. Today’s challenge is to solve the Add-A-Gram puzzle with Emacs/Elisp. I’ll give a solution next week.

Posted in Programming | Tagged , | 1 Comment

Weitz On Macros

Edi Weitz has a very nice set of notes on The Power of Lisp Macros. They’re from a talk he gave at freiheit.com and are aimed at the beginning Lisper. They’re amazingly easy to follow even though you aren’t hearing the accompanying commentary. It’s definitely worth a few minutes of your time.

Weitz is, of course, the author of much Common Lisp open source software including the indispensable CL-PPCRE that brings industrial strength Perl compatible regular expressions to Common Lisp. There’s lots of stuff on his site so it’s definitely worth a visit.

Posted in Programming | Tagged | Leave a comment

Org Mode And Living Documents

Vsevolod Dyomkin over at Lisp, The Universe and Everything has another in his series of interviews with Lisp hackers. This time it’s with Christophe Rhodes, the principal maintainer of SBCL. It’s an interesting interview and well worth a read so head on over.

One thing that Rhodes said that really resonated with me (probably because I feel the same) is this little bit about Org mode

“I have in the last couple of years fallen in love with org-mode and
in particular with “reproducible research”, or living documents; the
ability to have executable sections of code for generating results,
which can then themselves be further analysed by other bits of code —
in other languages, if that’s appropriate — and the whole exported as
a document typeset to what I consider to be high-quality standards.
The fact that there is also acceptable-quality web output from the
exact same process is gravy.”

There’s just something great about having a single document that contains all your notes, code to generate diagrams or compute results, and markup that can generate high quality typeset documents or HTML.

If any Emacs users out there have not yet tried Org mode, I urge you to do so without delay. There’s a compact guide to get you started as well as a full manual. Both are available as PDFs as well. Also, Bernt Hansen’s excellent Org Mode—Organize Your Life In Plain Text! is a great example of how to leverage Org mode. I found it very helpful as a go by when I was learning Org mode.

Posted in General | Tagged , | 1 Comment

The TSA (Again)

Yesterday I wrote that I don’t often post about the TSA but the latest news is just too good to pass up. Congress is holding hearings on the TSA so naturally they invited Bruce Schneier to testify. That only makes sense: he’s an acknowledged expert in the security field and has written widely on airport security and the TSA.

Except that the TSA was afraid of the heat and asked that he be removed from the witness list. The politicians, craven as always, complied.

Posted in General | Tagged , | Leave a comment

Huh?

Why the Programming Language C is Obsolete. C obsolete? Really? Someone forgot to tell Linus. Bjarne Stroustrup is clearly a smart guy and by all accounts a nice guy so I hate to call his baby ugly but Damn! that’s one ugly baby.

At one point in my career I tried really hard to like C++ but I could never convince myself that it was anything other than a disaster. Perhaps I’m not smart enough to learn C++ adequately or to see its manifest superiority because it still looks ugly to me. One advantage of writing in C rather than a Lisp-like language or Python or Ruby is that you can get almost down to the metal. Lots of fine grained control but, of course, you pay for that by having to worry about memory management and other annoying details. Another advantage, for me, is that I have an excellent mental model of what code will be generated for a given C statement and can use that model to pick between alternative C constructs for a particular task.

C++ adds complexity and destroys my mental model but still requires me to worry about low level details like memory management. How is this an improvement? Object Oriented Programming I hear you say. Except that it’s not clear that the benefits are worth the costs as far as C++ goes. I know, I know, Apostasy! Still, I can’t help but agree with Linus on this.

Posted in General | 1 Comment

You Can’t Make This Stuff Up

One of our interests here at Irreal is security so we are occasionally obligated to report on the doings of the TSA. The reason for this is that the TSA so often serves as an example of how not to do security.

I don’t write a lot of posts on the TSA because, after all, they’re all pretty much the same: the TSA does something so stupid and irrational that it beggars belief. Still, sometimes their idiocy just cries out for comment. Here’s the latest example.

Update: Removed superfluous “is”.

Posted in General | Tagged , , | Leave a comment

Note Taking With Org

I written a few posts recently about how I use Org mode to organize my projects and life (1, 2) so of course I was interested to see Da Zhang’s post on how he uses org-mode as the ultimate note taking tool. It turns out that he uses many of the same strategies that I do but he has some interesting additions.

For example, he exports each of his Org note files to HTML so he can view it in his browser. He also has a little bit of Elisp code that automates some of the tasks involved in taking notes such as adding the header information to the top of the file.

If you use Org mode and want some ideas on how to integrate it more tightly into your work flow, you should take a look at Zhang’s post.

Posted in General | Tagged , | 1 Comment

About Those Insecure RSA Keys

Last month I wrote a short post on some research that showed some RSA public keys are insecure. A couple days ago I noticed that the excellent Programming Praxis has a challenge based on that research and subsequent reporting in the press. This is excellent because it exposes how easy it was to gather the information and make the calculations.

There were two teams which did essentially the same thing: they used nmap to scan every IPv4 address on the Internet to discover which ones had ports 22 (ssh) or 448 (ssl) open and then used a custom piece of software to start the negotiation process and recover the SSL certificate and/or SSH host key of the site. According to this blog post by Nadia Heninger (one of the researchers) her team collected 10 million SSH host keys and 5.8 million SSL certificates. They discovered two problems:

  • Some keys were repeated meaning that compromising one site with a repeated key meant compromising all sites with that key.
  • Some keys shared one of the two primes that make up the public key.

It’s the second problem that concerns us here.

For those not familiar with the RSA algorithm, a public key consists of an exponent and a modulus N where N is the product of two large, random primes. The public key can be used to encrypt a message but in order to decrypt it, you need to know the factors of the modulus. What makes RSA secure is that it’s very hard to factor large numbers (RSA moduli are typically 2048 or 4096 bits long). However, if two moduli share a factor then it is easy to discover that factor by computing the greatest common denominator (gcd) of them. The gcd is one of the factors and the other factor of each key is trivially recovered by dividing by the gcd.

To show how simple this is, here is a solution to the problem presented by Programming Praxis

(defparameter rsa-keys '(708894553 704488409 705674273
                         707478271 710167019 708093251
                         702013379 704030867 708691187
                         708374743 712748719 713581951
                         704387447 707015741 704308279
                         710872423 707947453 704996429
                         706323757 705031051 714623803))

(defun check-for-common-factors (keys)
  (unless (null keys)
    (let ((key (car keys)))
      (dolist (k (cdr keys))
        (let ((g (gcd k key)))
          (when (> g 1)
            (format t "keys ~a and ~a have common factor ~a~%" key k g)
            (format t "~3t~a = (~a) (~a)~%" key g (/ key g))
            (format t "~3t~a = (~a) (~a)~%" k g (/ k g)))))
      (check-for-common-factors (cdr keys)))))

When we run that we get

CL-USER> (time (check-for-common-factors rsa-keys))
keys 704488409 and 706323757 have common factor 12401
   704488409 = (12401) (56809)
   706323757 = (12401) (56957)
keys 705674273 and 707015741 have common factor 12421
   705674273 = (12421) (56813)
   707015741 = (12421) (56921)
keys 707478271 and 708374743 have common factor 12451
   707478271 = (12451) (56821)
   708374743 = (12451) (56893)
keys 708093251 and 708691187 have common factor 12457
   708093251 = (12457) (56843)
   708691187 = (12457) (56891)
keys 704030867 and 704996429 have common factor 12379
   704030867 = (12379) (56873)
   704996429 = (12379) (56951)
keys 704387447 and 705031051 have common factor 12377
   704387447 = (12377) (56911)
   705031051 = (12377) (56963)
(CHECK-FOR-COMMON-FACTORS RSA-KEYS) took 353 microseconds (0.000353 seconds) to run 
                    with 4 available CPU cores.
During that period, 350 microseconds (0.000350 seconds) were spent in user mode
                    8 microseconds (0.000008 seconds) were spent in system mode
 13,920 bytes of memory allocated.

In a sense, this is a toy problem because

  • The “rsa-keys” are too small to be an actual key—they could easily be factored directly.
  • There are only 21 keys in the sample, not 15 million as in the actual data.

The first of these isn’t really a problem because the Lisp bignum package can easily handle numbers the size of real keys. The second issue is a problem because check-for-common-factors is a O(n2) algorithm. Fortunately, it’s possible to use a linear algorithm (it’s described in the Programming Praxis post) but it doesn’t add anything to our understanding, it just makes things run faster (Heninger estimates that using the algorithm I showed would take about 24 years; using the (almost) linear method processed the 5.8 million SSL certificates in 18 hours).

Be sure to read Heninger’s post. It’s short and gives an excellent overview of the research and what it all means.

Posted in Programming | Tagged , | Leave a comment