Bill Joy’s Greatest Gift To Man

Posted on 2012-01-05 by jcs

That would be the vi editor, of course. For some reason the folks over at Hacker News have been posting some old interviews with Bill Joy about the vi editor. In Bill Joy’s greatest gift to man—the vi editor, Ashlee Vance of The Register posits that even more than BSD Unix, NFS, or the UltraSPARC, Joy’s greatest accomplishment is vi. Certainly there’s a lot of people who think so. The rest of the article is an interview of Joy that explores the history of vi a bit.

Steve Kirkendall points to a 1984 Interview with Bill Joy from the old Unix Review magazine that discusses the history of vi in a bit more detail. Both articles are interesting and worth a read.

I used vi (and vim) for many years and still have warm feelings for it. It’s a great editor and along with Emacs (and arguably TextMate) one of the only 2 or 3 editors that serious developers use. I switched to Emacs when I started using Lisp and Scheme seriously (as described here and here) and wouldn’t change back but I still have a lot of respect for vi.

Posted in General | 1 Comment

File Loading In Emacs

Posted on 2012-01-04 by jcs

Xah Lee has a very nice post up that explores Emacs Lisp’s Library System and what require, load, load-file, autoload, and feature really do and how they differ. This is one of those subject areas that most of us just absorb by osmosis as we learn Elisp so it’s nice to have it written down and made explicit. If you’re an Elisp programmer, be sure to give it a read.

Posted in Programming | Tagged , | Leave a comment

Password Cluelessness

Posted on 2012-01-03 by jcs

Over at self.li, Peter Legierski has a horrifying post about password cluelessness. The tl;dr is that fon, described as the “world’s largest Wi-Fi Network” keeps its passwords as plain text. Legierski knows this because when he forgot his password and clicked on the appropriate link, they sent it to him in an email.

We’ve discussed this sort of thing many times before here at Irreal. Even putting aside the massive fail that is sending a password via email in the clear, the storing of passwords as plain text is just not acceptable. They absolutely must be salted and hashed to have even a modicum of security. Fon is a community with over 4 million users each of whom makes his WiFi router available to members so that every member always has access to free WiFi wherever they travel. How many of those 4 million users do you think reuse their passwords? What would be the consequences of the disclosure of that information?

As I said, we’ve discussed all this before so there wouldn’t be much reason to beat the remains of that particular horse except for the comments to Legierski’s post. The amount of cluelessness displayed (at full volume) is extraordinary. Many commenters said that perhaps the passwords were held in an encrypted database, which would make it OK. Others thought that you can “decrypt” a hash with a rainbow table and recover the password. Some appeared not to have heard about SSL/TLS. If you’re interested in this sort of thing, you really should follow the link and read the comments.

Sadly, this cluelessness appears to be endemic. One of the commenters, Igal Tabachnik, notes that the storing of passwords as plain text is so widespread that he started a Web site, Plain Text Offenders, to document and shame the offenders. The comments to Legierski’s post show that many users, even nominally technical ones, don’t have a clue about the proper handling of passwords. Tabachnik’s site shows that many programmers and IT people don’t either.

Posted in General | Tagged | Leave a comment

The Emacs compile-command Local Variable

Posted on 2012-01-02 by jcs

I just ran across this gem in an old Justinhj’s Coding Blog post. Most of the time developers are working on projects that are complex enough to require a make file and for them, Emacs does the right thing when you call 【Meta+x】 compile by using make -k as the compile command. Sometimes, though, you have a simple, single file that you’d like to compile and for those you have to type in the compile command by hand after the 【Meta+x】 compile.

As Justin points out, there’s a better way. You can set the local file variable compile-command in the source file to the correct compile command. For example, I added this line to the top of the makepw.c file (that I wrote about here) to control the compilation. 

/* -*- compile-command: "gcc -Wall -lcrypto -o makepw makepw.c" -*- */

That works very nicely and any time I make changes or need to recompile it for some reason I can just type 【Meta+x】 compile and Emacs does the rest for me.

One caveat: Emacs searches for and evaluates file local variables when the file is visited, so if you are starting from scratch you will need to do something like save the file and then do a 【Meta+x】 revert-buffer to get the variable evaluated. After that, Emacs will set it each time you load the file. It also has the additional benefit of documenting the proper compiler invocation, which may not be obvious because of library dependencies and so on.

I didn’t know this trick and Justin’s post was from 2009 so it’s worthwhile pointing it out again for people like me who weren’t aware of the compile-command local file variable.

Update: Most of time → Most of the time; worth while → worthwhile

Posted in Programming | Tagged | 3 Comments

Emacs Syntax Classes

Posted on 2011-12-31 by jcs

This is sort of a note to myself. I sometimes need to know the character designation of a syntax class. Most often this is for use with the \sC syntax in a regexp. For example 

(looking-at "^\\s-*$\\|\\s-*;")

asks if we’re looking at a line containing only whitespace or a line with possible leading whitespace and then a ;. (It’s used to recognize lines in a Scheme program file with no executable code.)

The problem is I always have a hard time finding the Syntax Class Table in the Elisp manual. Xah Lee has helpfully provided a bunch of pages like HTML/XML Entities List, Unicode Space Symbols, and a Regexp Cheat Sheet that I’ve bookmarked for easy reference but nothing for the syntax table. Therefore, I’m making this short table mostly so I can bookmark it.

The characters in a given syntax class usually depend on the current major mode so the table doesn’t give the full story. For that you need to look at the Syntax Class Table node of the Elisp manual.

Syntax Class Character Designation
Whitespace Character - (hyphen) or ␠
Word Constituent w
Symbol Constituent _ (underscore)
Punctuation Character .
Open Parenthesis (
Close Parenthesis )
String Quote "
Escape-Syntax Character \
Character Quote /
Paired Delimeter (TeX $) $
Expression Prefix '
Comment Starter <
Comment Ender >
Inherit Standard Syntax @
Generic Comment Delimiter !
Generic String Delimiter |

Update: Designated hyphen and underscore explicitly.

Posted in Programming | Tagged , | 3 Comments

The Emacs regexp-opt Function

Posted on 2011-12-29 by jcs

Sacha Chua introduced me to regexp-opt, an Emacs function that I hadn’t seen before. The idea is that regexp-opt will take a list of strings and return a more or less optimal regexp to recognize any of those strings.

Because it does analysis and restructuring of the list of strings, regexp-opt will generally produce a more efficient regexp than the equivalent 

(mapconcat 'regexp-quote strings "\\|")

In addition to the list of strings, regexp-opt takes an optional second argument that if not nil will surround the regexp with grouping parentheses. If the second argument is 'words, then regexp-opt also surrounds the regexp with \\< and \\>.

Here’s an example from the source code for regexp-opt that shows how it optimizes a regexp. 

(let ((strings '("cond" "if" "when" "unless" "while"
                    "let" "let*" "progn" "prog1" "prog2"
                    "save-restriction" "save-excursion" "save-window-excursion"
                    "save-current-buffer" "save-match-data"
                    "catch" "throw" "unwind-protect" "condition-case")))
  (concat "(" (regexp-opt strings t) "\\>"))
 => "(\\(c\\(atch\\|ond\\(ition-case\\)?\\)\\|if\\|let\\*?\\|prog[12n]\\|save-\\(current-buffer\\|excursion\\|match-data\\|restriction\\|window-excursion\\)\\|throw\\|un\\(less\\|wind-protect\\)\\|wh\\(en\\|ile\\)\\)\\>"

This regexp takes approximately two-thirds the amount of time that the equivalent mapconcat regexp does.

Simon Marshall, the code’s author, notes that the package was written to produce efficient regexps, not to produce regexps efficiently and therefore you should avoid using it in-line too much unless you use eval-when-compile. See the commentary in regexp-opt.el for more on this. You can, of course, easily get to the code from the function description by clicking on the file name.

Update: Fixed mapconcat example.

Posted in Programming | Tagged , | Leave a comment

Solution To The Emacs Programming Challenge

Posted on 2011-12-28 by jcs

A few days ago, I posed an Elisp programming challenge that asked you to make the record 

(record
  (date "2005-02-21T18:57:39")
  (millis 1109041059800)
  (sequence 1)
  (logger nil)
  (level 'SEVERE)
  (class "java.util.logging.LogManager$RootLogger")
  (method 'log)
  (thread 10)
  (emessage "A very very bad thing has happened!")
  (exception
    (emessage "java.lang.Exception")
    (frame
      (class "logtest")
      (method 'main)
      (line 30))))

executable in such a way that it transforms itself into the equivalent XML record when executed.

On the day I posed the challenge, I woke up thinking that making the record transform itself into XML was the proper way to turn it into XML rather than the way I described in Converting S-Expression To XML In Emacs. It seemed like a direct application of the ideas in my More Fun With Log Files Stored As Lisp post and I decided to program it just for fun with no idea of turning it into a post.

A couple of hours later I was still flailing around so maybe the application wasn’t as direct as I thought. Actually, the idea is the same but because you’re not transforming it back into Lisp it’s a little trickier.

In any event, here’s how I solved the problem. First define all the tags except record to be functions of the form 

(defun date (&rest args) (toxml 'date args))

As I explained in the Exploratory Programming In Emacs post that involved little more than copying the list of tags and a keyboard macro. The toxml function is 

(defun toxml (tag args)
  (with-output-to-string
    (princ (format "\n<%s>" tag))
    (mapc 'princ args)
    (princ (format "</%s>" tag))))

The idea is that each sexpr returns a string of the arguments wrapped in <tag>…</tag> where tag is the symbol in the function slot of the sexpr. That is, (date "2005-02-21T18:57:39") is transformed to "<date>2005-02-21T18:57:39</date>". If we temporarily alias record to list and executed the record sexpr, we get 

("
<date>2005-02-21T18:57:39</date>" "
<millis>1109041059800</millis>" "
<sequence>1</sequence>" "
<logger>nil</logger>" "
<level>SEVERE</level>" "
<class>java.util.logging.LogManager$RootLogger</class>" "
<method>log</method>" "
<thread>10</thread>" "
<emessage>A very very bad thing has happened!</emessage>" "
<exception>
<emessage>java.lang.Exception</emessage>
<frame>
<class>logtest</class>
<method>main</method>
<line>30</line></frame></exception>")

which shows what the record function will see when it gets called. Note that each string begins with a newline so the ” ” at the end of each line is a close quote followed by an open quote.

Finally, we need to define a record function to stitch the strings together and get rid of that annoying nil in <logger>nil</logger>. 

(defun record (&rest args)
  (princ "<record>")
  (princ (replace-regexp-in-string ">nil<" "><" (apply 'concat args)))
  (princ "</record>"))

The princ on the third line gets rid of the quote marks that the concat places around the final result of the apply.

Now when we execute the record we get 

<record>
<date>2005-02-21T18:57:39</date>
<millis>1109041059800</millis>
<sequence>1</sequence>
<logger></logger>
<level>SEVERE</level>
<class>java.util.logging.LogManager$RootLogger</class>
<method>log</method>
<thread>10</thread>
<emessage>A very very bad thing has happened!</emessage>
<exception>
<emessage>java.lang.Exception</emessage>
<frame>
<class>logtest</class>
<method>main</method>
<line>30</line></frame></exception></record>

Update: Let me mention again that the inspiration for this series of posts and the sample record sexpr are from Steve Yegge’s The Emacs Problem. Be sure to give it a read if you haven’t already.

Posted in Programming | Tagged , | 3 Comments

Turnkey Publishing

Posted on 2011-12-27 by jcs

I’ve written several posts about the challenges facing the publishing industry and how their continued existence—at least in their current form—is looking increasingly tenuous. Just today Charlie Stross described the Internet as a “communication tool that tends to disintermediate supply and demand” and, sadly, disintermediation is biggest threat to publishers.

Jason Crawford has a provocative post entitled Startup idea: Turnkey self-publishing service for authors. In it he tells the story of a friend who self published a book and the things he had to do as his own publisher. Then Crawford throws out a startup idea: Provide a turnkey, fee-for-service publishing service. It would be pretty much like a typical publisher except that the author would pay an up front fee and then keep all the profits from the book.

Yes, there are problems to be solved with this idea but it’s probably not as hard as you might imagine. In the first place, almost everything a publisher does is farmed out: copy editors, technical editors, typesetting, printing, and so on. These people are already free-lance and would, I’m sure, be just as happy to work for a service such as this as they are to work for traditional publishers.

This idea, even if it comes to fruition, isn’t, by itself, a lethal threat to publishers but when you add in all the other threats that I’ve written about before, the threat seems more immediate.

Posted in General | Tagged | 1 Comment

WordPress And Trackbacks

Posted on 2011-12-26 by jcs

I’m a big fan of WordPress. I know a lot of people disagree but it works very well for me. It allows me to maintain a nicely laid out blog without spending a lot of time with administration. I do almost everything from Emacs and seldom have to go to the administration panel of the blog. Except to deal with spam.

After I reluctantly installed nucaptcha, comment spam ceased to be a problem until the spammers started injecting their links via trackbacks. Once again, I reluctantly reduced the user experience by turning off trackbacks but I noticed that I was still getting occasional trackback spam. Lately, the number of trackback spams have increased so I spent a little time tracking down the problem.

It turns out that when you turn off trackbacks it affects only future posts but leaves it enabled for existing posts. The spammers, of course, don’t care because the point is to get the links into the posts for Google to see not for anyone to read. Thus they were simply adding trackbacks to my older posts.

There are a couple of problems here. First, to my way of thinking, the principle of least surprise dictates that when you turn off trackbacks, WordPress should, you know, turn off trackbacks. There’s no indication that only new posts will be affected, just a check box to enable or disable them. I understand why it works the way it does (WordPress can simply set each post to allow or disallow trackbacks as they’re posted) but it isn’t really that more difficult to spin through the database and turn it off for all posts.

Instead, it has to be done by hand1. First of all, it isn’t easy to figure out how to do that because the option is hidden by default (I know, I know, RTFM but again I use WordPress so I don’t have to spend a lot of time reading manuals or doing administrative things). Then for each old post you have to disable trackbacks and then update the post. No big thing for a single post but a significant amount of work for 300 posts.

I’m working my way through the old posts so in a few days the blog should be spam-free again—at least until the spammers introduce their next trick. I just wish it were easier.

Footnotes:

1 Yes, I know about plugins to do this sort of thing but every plugin is another potential security vulnerability and it’s more administrative effort ongoing.

Posted in Blogging | Tagged | Leave a comment

Emacs’ jump-to-register In Exploratory Programming

Posted on 2011-12-25 by jcs

I’ve written before about Emacs Registers and how you can store the mark in a register with 【Ctrl+x r Space】 register. That’s one of those things that’s occasionally useful because you can then jump to that position with 【Ctrl+x r j】 register but in my experience, I just don’t use it that often. However, while I was doing exploratory programming for my series on executable log records, I found that it was extraordinarily useful.

As I explained in the exploratory programming post, I developed the code by writing some exploratory functions and then trying them out on a record sexpr. I used the same file for the entire series and as I added additional posts to the series, I just added code to the end of the buffer. Then I would try it out on one of the records that were at the top of the buffer. By storing the mark in a register, I was able to quickly return to the end of the record and execute it with 【Ctrl+x Ctrl+e】. Then I could return to the code I was working on by simply going to the end of the buffer with 【Meta+>】.

None of this is extraordinary or ground breaking but it does show how a simple Emacs command can really speed up development. Without storing the position of the record in a register, I would have had to move to it with a much more complicated series of keystrokes. This is another example of Emacs providing the very tool you need for a specific task.

Posted in Programming | Tagged | Leave a comment