Org Mode And Living Documents

Vsevolod Dyomkin over at Lisp, The Universe and Everything has another in his series of interviews with Lisp hackers. This time it’s with Christophe Rhodes, the principal maintainer of SBCL. It’s an interesting interview and well worth a read so head on over.

One thing that Rhodes said that really resonated with me (probably because I feel the same) is this little bit about Org mode

“I have in the last couple of years fallen in love with org-mode and
in particular with “reproducible research”, or living documents; the
ability to have executable sections of code for generating results,
which can then themselves be further analysed by other bits of code —
in other languages, if that’s appropriate — and the whole exported as
a document typeset to what I consider to be high-quality standards.
The fact that there is also acceptable-quality web output from the
exact same process is gravy.”

There’s just something great about having a single document that contains all your notes, code to generate diagrams or compute results, and markup that can generate high quality typeset documents or HTML.

If any Emacs users out there have not yet tried Org mode, I urge you to do so without delay. There’s a compact guide to get you started as well as a full manual. Both are available as PDFs as well. Also, Bernt Hansen’s excellent Org Mode—Organize Your Life In Plain Text! is a great example of how to leverage Org mode. I found it very helpful as a go by when I was learning Org mode.

Posted in General | Tagged , | 1 Comment

The TSA (Again)

Yesterday I wrote that I don’t often post about the TSA but the latest news is just too good to pass up. Congress is holding hearings on the TSA so naturally they invited Bruce Schneier to testify. That only makes sense: he’s an acknowledged expert in the security field and has written widely on airport security and the TSA.

Except that the TSA was afraid of the heat and asked that he be removed from the witness list. The politicians, craven as always, complied.

Posted in General | Tagged , | Leave a comment

Huh?

Why the Programming Language C is Obsolete. C obsolete? Really? Someone forgot to tell Linus. Bjarne Stroustrup is clearly a smart guy and by all accounts a nice guy so I hate to call his baby ugly but Damn! that’s one ugly baby.

At one point in my career I tried really hard to like C++ but I could never convince myself that it was anything other than a disaster. Perhaps I’m not smart enough to learn C++ adequately or to see its manifest superiority because it still looks ugly to me. One advantage of writing in C rather than a Lisp-like language or Python or Ruby is that you can get almost down to the metal. Lots of fine grained control but, of course, you pay for that by having to worry about memory management and other annoying details. Another advantage, for me, is that I have an excellent mental model of what code will be generated for a given C statement and can use that model to pick between alternative C constructs for a particular task.

C++ adds complexity and destroys my mental model but still requires me to worry about low level details like memory management. How is this an improvement? Object Oriented Programming I hear you say. Except that it’s not clear that the benefits are worth the costs as far as C++ goes. I know, I know, Apostasy! Still, I can’t help but agree with Linus on this.

Posted in General | 1 Comment

You Can’t Make This Stuff Up

One of our interests here at Irreal is security so we are occasionally obligated to report on the doings of the TSA. The reason for this is that the TSA so often serves as an example of how not to do security.

I don’t write a lot of posts on the TSA because, after all, they’re all pretty much the same: the TSA does something so stupid and irrational that it beggars belief. Still, sometimes their idiocy just cries out for comment. Here’s the latest example.

Update: Removed superfluous “is”.

Posted in General | Tagged , , | Leave a comment

Note Taking With Org

I written a few posts recently about how I use Org mode to organize my projects and life (1, 2) so of course I was interested to see Da Zhang’s post on how he uses org-mode as the ultimate note taking tool. It turns out that he uses many of the same strategies that I do but he has some interesting additions.

For example, he exports each of his Org note files to HTML so he can view it in his browser. He also has a little bit of Elisp code that automates some of the tasks involved in taking notes such as adding the header information to the top of the file.

If you use Org mode and want some ideas on how to integrate it more tightly into your work flow, you should take a look at Zhang’s post.

Posted in General | Tagged , | 1 Comment

About Those Insecure RSA Keys

Last month I wrote a short post on some research that showed some RSA public keys are insecure. A couple days ago I noticed that the excellent Programming Praxis has a challenge based on that research and subsequent reporting in the press. This is excellent because it exposes how easy it was to gather the information and make the calculations.

There were two teams which did essentially the same thing: they used nmap to scan every IPv4 address on the Internet to discover which ones had ports 22 (ssh) or 448 (ssl) open and then used a custom piece of software to start the negotiation process and recover the SSL certificate and/or SSH host key of the site. According to this blog post by Nadia Heninger (one of the researchers) her team collected 10 million SSH host keys and 5.8 million SSL certificates. They discovered two problems:

  • Some keys were repeated meaning that compromising one site with a repeated key meant compromising all sites with that key.
  • Some keys shared one of the two primes that make up the public key.

It’s the second problem that concerns us here.

For those not familiar with the RSA algorithm, a public key consists of an exponent and a modulus N where N is the product of two large, random primes. The public key can be used to encrypt a message but in order to decrypt it, you need to know the factors of the modulus. What makes RSA secure is that it’s very hard to factor large numbers (RSA moduli are typically 2048 or 4096 bits long). However, if two moduli share a factor then it is easy to discover that factor by computing the greatest common denominator (gcd) of them. The gcd is one of the factors and the other factor of each key is trivially recovered by dividing by the gcd.

To show how simple this is, here is a solution to the problem presented by Programming Praxis

(defparameter rsa-keys '(708894553 704488409 705674273
                         707478271 710167019 708093251
                         702013379 704030867 708691187
                         708374743 712748719 713581951
                         704387447 707015741 704308279
                         710872423 707947453 704996429
                         706323757 705031051 714623803))

(defun check-for-common-factors (keys)
  (unless (null keys)
    (let ((key (car keys)))
      (dolist (k (cdr keys))
        (let ((g (gcd k key)))
          (when (> g 1)
            (format t "keys ~a and ~a have common factor ~a~%" key k g)
            (format t "~3t~a = (~a) (~a)~%" key g (/ key g))
            (format t "~3t~a = (~a) (~a)~%" k g (/ k g)))))
      (check-for-common-factors (cdr keys)))))

When we run that we get

CL-USER> (time (check-for-common-factors rsa-keys))
keys 704488409 and 706323757 have common factor 12401
   704488409 = (12401) (56809)
   706323757 = (12401) (56957)
keys 705674273 and 707015741 have common factor 12421
   705674273 = (12421) (56813)
   707015741 = (12421) (56921)
keys 707478271 and 708374743 have common factor 12451
   707478271 = (12451) (56821)
   708374743 = (12451) (56893)
keys 708093251 and 708691187 have common factor 12457
   708093251 = (12457) (56843)
   708691187 = (12457) (56891)
keys 704030867 and 704996429 have common factor 12379
   704030867 = (12379) (56873)
   704996429 = (12379) (56951)
keys 704387447 and 705031051 have common factor 12377
   704387447 = (12377) (56911)
   705031051 = (12377) (56963)
(CHECK-FOR-COMMON-FACTORS RSA-KEYS) took 353 microseconds (0.000353 seconds) to run 
                    with 4 available CPU cores.
During that period, 350 microseconds (0.000350 seconds) were spent in user mode
                    8 microseconds (0.000008 seconds) were spent in system mode
 13,920 bytes of memory allocated.

In a sense, this is a toy problem because

  • The “rsa-keys” are too small to be an actual key—they could easily be factored directly.
  • There are only 21 keys in the sample, not 15 million as in the actual data.

The first of these isn’t really a problem because the Lisp bignum package can easily handle numbers the size of real keys. The second issue is a problem because check-for-common-factors is a O(n2) algorithm. Fortunately, it’s possible to use a linear algorithm (it’s described in the Programming Praxis post) but it doesn’t add anything to our understanding, it just makes things run faster (Heninger estimates that using the algorithm I showed would take about 24 years; using the (almost) linear method processed the 5.8 million SSL certificates in 18 hours).

Be sure to read Heninger’s post. It’s short and gives an excellent overview of the research and what it all means.

Posted in Programming | Tagged , | Leave a comment

Sending Mail Semi-Programmatically In Emacs

I have an ongoing project that I manage with an Org mode file. The file has the structure

* [2012-03-21 Wed]
** Second Level Headline
   Some information about the second level headline
...

where the level one headlines are the date and the level two headlines describe an event that happened on that date.

Most days, there are two or three events and every time I enter one I email a teammate with just the second level headline. I used to do that by simply retyping the headline into an email or maybe cutting and pasting. Then I wondered if I could automate that a bit. Here’s what I can up with:

(defun mail-org-headline ()
  (interactive)
  (let* ((bnds (bounds-of-thing-at-point 'line))
         (line (buffer-substring-no-properties (car bnds) (cdr bnds))))
   (with-temp-buffer
     (insert "To: someone@somewhere.com\nSubject: Headline Report\n\n")
     (insert (substring line 3))
     (mail-send))))

With the point on the headline I want to send, I call mail-org-headline and the email is automatically prepared and pops up in the OS X Mail App ready to send. Because the email is prepared in a temporary buffer, there is nothing to clean up in Emacs.

The call to substring on the next-to-last line strips off the two stars and space. Since I use the native Mail.app on my Mac, I’ve never bothered configuring the Emacs mail apparatus. If I did, I could probably get Emacs to send the email directly without invoking Mail but I like it the way it is because it gives me a chance to check the email before I send it.

I don’t imagine many people have the same use case as I described above but I decided it was worth sharing to show how to send mail programmatically from Emacs. If you want to do something similar, be aware that you can use most of the usual header fields in addition to To: and Subject:.

Posted in Programming | Tagged , , | 3 Comments

Org-Drill

When I was browsing Hacker News today, I stumbled upon a neat Org mode extension called org-drill. The idea is that it implements an electronic version of flash cards that are useful for learning vocabulary or other facts. Obviously, this is mostly useful to students but anyone who is trying to learn a set of related facts would probably find it useful.

This is a pretty sophisticated package that uses a spaced repetition algorithm to optimize learning the material and that has several different modes for presenting the questions and answers. The questions and answers are, of course, plain text org files so they’re easy to set up and modify. The page at the above link discusses the different modes in detail and is worth taking a look at if you think you might be interested in something like this.

Setup is a breeze. The org-drill.el file is part of the contributed packages so you just need to add a pointer to that directory to your Emacs load-path if you don’t already have one and then add

(require 'org-drill)

to your .emacs or init.el file. You can also enable it with the customize interface if you prefer. The details are on the org-drill page.

The org-drill package is another example of the power and flexibility of Emacs and Org mode.

Posted in General | Tagged , | Leave a comment

Org-Mac-Link-Grabber

Yesterday I wrote about my on-going effort to get rid of paper and to keep all my records and projects anchored in Org files. Yesterday’s post was about easily linking to receipts from my tax file. That’s nice because I can open the Org file for this year’s taxes and if I need information about the receipt for an item I can just click on the link and bring up a PDF of the receipt.

As part of my efforts to keep everything together, I also keep a log in the tax file where I log the date and time for any event having to do with taxes. For example, when I send the original documents to the accountant I log that and perhaps the UPS tracking number. When I get an email from the accountant, I log the date and time I got it and what it was about but I didn’t have a good way of recording a link to the email itself.

I remembered reading on the Org Mode Web Site about some utilities for Mac OS X that allowed linking and importing documents from various Mac Apps like Mail, iCal, Safari, and some others. It turns out that they are in the contrib directory of the distribution so all I had to do was add ~/tools/org-mode/contrib/lisp to the load-path list, add the following to init.el

(require 'org-mac-link-grabber)
(add-hook 'org-mode-hook
          (lambda ()
            (flyspell-mode 1)  ; <-- These two lines already existed.
            (abbrev-mode 1)    ; <-- They're not needed for omlg-grab-link
            (define-key org-mode-map (kbd "C-c g") 'omlg-grab-link)))

and call org-reload. Now when I want to grab a link to a mail message or a Web page, I just type 【Ctrl+c g】 and I get the menu

[F]inder [m]ail [a]ddressbook [s]afari [f]irefox [c]hrome:

in the minibuffer. When I choose an application, a link is inserted that points to the current selection in that application.

The org-mac-link-grabber page says that the following applications are supported:

  • Finder
  • Mail
  • Address Book
  • Safari
  • Firefox
  • Firefox with the Vimperator plugin
  • Chrome
  • Together

You can customize which applications appear on the menu.

If you have a Mac and use Org mode, you should definitely look into enabling this functionality. If all you care about is pointers to mail, there is another package org-mac-message that deals just with mail and is already in Org core so all you have to do is enable it as described on its page.

I’ve described this functionality in terms of keeping track of tax information but I anticipate that as I move my records and projects to Org, I will find many uses for it.

Update: (add-to-list ‘org-modules ‘(org-mac-link-grabber)) → (require ‘org-mac-link-grabber)

Posted in General | Tagged , , | 2 Comments

Spam: There’s A Lot Of That Going Around

Last week I wrote about a spike in forum spam that Irreal is seeing (1, 2). Of course, Irreal is a small site and even on our worst days we see at most 10–15 spam comments.

Charlie Stross, a noted SF author and proprietor of Charlie’s Diary, knows what a real spam problem looks like. After getting over 55,000 spams in the last two weeks, he’s restricted comments to the newest 5 posts and turned off commenting altogether for a period of time hoping that the spam bots will forget about him.

I don’t know why we’re seeing this sudden increase or even if it’s affecting more than a handful of sites. If you have a blog and have noticed an increase too, leave a comment.

Posted in General | Tagged | Leave a comment