What Makes Emacs So Great?

Posted on 2012-01-26 by jcs

Over at The Setup they are featuring Phil Hagelberg who has an especially nice take on what makes Emacs so great. He says that Emacs is as close as you can get on modern systems to the fully dynamic environment of the Lisp Machines of the 80’s. Almost any part of the environment can be changed at runtime without recompiling or restarting.

I think that’s exactly right. There are, I suppose, Emacs users who don’t change anything from the default setup but as Irreal readers know, I have all sorts of customizations to my Emacs and I am not in the least unique in this. One of the nice things about this is that I wrote those customizations using the same mechanisms as the authors of Emacs itself. Hagelberg says, “It’s hard to overstate the benefits of this setup.” It’s something that you get “for free” because when we make extensions to Emacs we’re writing in Lisp and as Doug Hoyte says in Let Over Lambda, “…lisp gives you the same tools available to the people who created your programming environment.”

Hagelberg goes on to say that this reduces the friction of tweaking your environment to such a low level that you’re more willing to experiment and improve things than you would be in a more standard environment. For example, I use OS X and really love it but it’s too hard to make anything other than fairly simple changes to the default behavior and most of those are changes that the system implementors anticipated and provided some way of effecting. I would guess that this ability to customize things is the reason that many Emacs users try to avoid leaving Emacs at all, choosing to read and send mail, use IRC and IM, listen to music, and many other things all from Emacs. Why not? You can pretty much have it anyway you want it.

Posted in General | Tagged , | Leave a comment

Demand Curves By Age

Posted on 2012-01-25 by jcs

H S Dent has a set of endlessly fascinating demand curves that show the demand for various kinds of consumer goods by the age of the consumer. Many of these are pretty much what you’d expect: demand for bicycles, for example, grows until about age 40 and then drops off to nothing a little after age 80. Still, there’s a lot of surprises.

Children’s clothing (of all sorts) drops off with increasing age as you’d expect but then rises again at a little after age 80. You see this again and again in everything from boys’ underwear to girls uniforms. I have no idea why this is. Oddly, adult clothing doesn’t follow the same pattern: you generally see the expected drop off with increasing age. Not all the curves show an increase to some age and then a decrease. Some rise and fall repeatedly throughout the age span—college tuition is an example of this. Some, like safe deposit box rental and prescription drugs shows a steady increase throughout the age span.

I doubt any of this is of particular importance to the typical Irreal reader but I find the graphs entertaining and in some cases puzzling. It well worth a look—you may be surprised at how interesting they can be.

Posted in General | Leave a comment

Russ Cox: Part 4 Of A Trilogy

Posted on 2012-01-24 by jcs

As long time readers know, I am a big fan of Russ Cox’s work. In particular, his 3 part series on implementing regular expression search is just outstanding. Now Cox has added a fourth article to the series that describes how he implemented Google Code Search.

There is, of course, too much code on the Web to do a grep over the entire corpus. Instead, they built a trigram index of the documents and then analyzed each regexp to decide which trigrams would need to be present in a document to satisfy the regexp. Then they used the trigram index to retrieve and grep just those documents. Cox’s paper discusses how they did the analysis—it more complicated than you might imagine—and presents some results on a smaller version for which the code is available. (Google discontinued Code Search so the standalone version is all there is).

One example is searching for “hello world” in the 3.1.3 Linux kernel. There are 36,972 files to search but only 25 of them contain the necessary trigrams: “ wo”, “ell”, “hel”, “llo”, “lo ”, “o w”, “orl”, “rld”, “wor”. That cuts the search time by 2 orders of magnitude.

This paper is a beautiful example of using some elementary data structures and some smart analysis to bring a huge problem down to a manageable size. I really recommend the article for anyone interested in programming.

Posted in General | Leave a comment

JSON And S-Expressions Redux

Posted on 2012-01-23 by jcs

After yesterday’s post on JSON Versus S-Expressions I thought that one obvious solution for people who want to use JSON tools to process logs is to have the (Lisp-based) logs convert themselves to JSON much like we did for S-expressions to XML. I thought about issuing another challenge to write Elisp code to do the conversion but it’s so similar to the Sexpr-to-XML example that I decided not to.

I’m pretty sure that I remember seeing Lisp code that converts JSON into Sexprs and vice versa so you could start with either Sexprs or JSON and easily convert to the other if that was more convenient. I think the code I saw was for either Scheme or CL so a good exercise would be to write Elisp code that converts JSON into Sexprs. That way you could work with JSON in Emacs.

Actually, that seemed like such a good idea that I thought someone must surely have done it and, of course, someone has. It’s even included with Emacs. Is there anything Emacs can’t do?

Posted in Programming | Tagged , , | Leave a comment

More On JSON Versus S-Expressions

Posted on 2012-01-22 by jcs

I’ve written several posts on JSON, most recently here and here. These last two posts considered the advantages of JSON over S-expressions for writing log files. Although S-expressions are more powerful and flexible than JSON, as explained in the 7-part series referenced in the Writing Log Files In JSON post, I concluded that JSON might, nevertheless, be the right answer because there are tools available for working with JSON in almost every language, whereas S-expressions, with a few exceptions, are most naturally handled with some flavor of Lisp.

Now, a post on the SHiNKiROU Blog entitle S-Expressions: The Fat-Free Alternative to JSON is giving me second thoughts. The context of the post is a bit larger than log files and includes such things as document markups. JSON is certainly a win over XML for key-value-pairs, the post says, but not so good for things like HTML markup. S-expressions, on the other hand, can do both jobs equally well. Chapter 16 Paul Graham’s ANSI Common LISP, for example, describes a set of functions for turning S-expressions into HTML.

I find the post pretty persuasive although I do still worry about non-Lispers being comfortable with sexprs or at least being able to do useful things with them right off the bat. Say what you will about JSON, it’s very easy to learn just about everything you need to know about them—excluding the various tools that work on them, of course. S-expressions are pretty easy too but I don’t think there are as many (non-Lisp) tools available to work with them. Much as I would like everyone to learn a bit of Lisp, we all know that’s not going to happen so where does that leave us? Should we use the more powerful and simple S-expressions, or should be use JSON because of its multi-language support? What do you think?

Posted in General | Tagged , | 1 Comment

PostgreSQL 9.2 To Support JSON As A Core Type

Posted on 2012-01-21 by jcs

Andrew Dunstan has a post in his Andrew’s PostgreSQL blog reporting that PostgreSQL 9.2 will support JSON as a core type. This is yet another reason to favor JSON as a universal log reporting format as I discussed in my Writing Log Files In JSON post.

Of course, as a Lisper I’d still use S-expressions for any of my own projects due to their flexibility and self-transforming capabilities. Still, if you’re going to have non-Lispers using those logs, JSON makes a lot of sense. Certainly more than the common “printf” style logs, and especially more than an XML abomination.

Posted in General | Leave a comment

Functional Thinking

Posted on 2012-01-20 by jcs

Here’s an interesting talk about thinking in a functional way by Neal Ford. I’ve tagged this post as Lisp but Ford doesn’t use Lisp—he uses some weird mutation of Java, a little Groovy, and a little Scala. The point of the talk, he says, is not about functional programming languages but about learning to think functionally.

Be sure not expand the embedded video because the slides appear in the bigger box under the video.

Posted in Programming | Tagged | Leave a comment

Useful Emacs Commands Reference

Posted on 2012-01-19 by jcs

Xah Lee has several useful reference pages, such as this one of HTML/XML Entities, that I use all the time. I have them bookmarked for easy reference so that when I, for example, need to know the Unicode character for ρ, I can just bring up the page in my browser to look it up. I’ve also been bookmarking pages from my blog that I describe as “notes to myself” for the same reason.

Over time I find that I have a bunch of pages with a lot of explanatory text and then a summary at the end that is what I’m really interested in. I decided to combine those summaries into a single document to make things easier to find. Since those notes-to-myself posts tended to generate a lot of comments (by Irreal standards) I thought I would make the page available in case anyone else found it useful.

The page is here if you’d like to bookmark it and I’ve also added a link to it over on the right under Links.

Posted in General | Tagged | 5 Comments

Emacs On Google+ And Strange Keybindings

Posted on 2012-01-18 by jcs

Xah Lee announced on his Emacs Blog that he has started an Emacs stream on G+. If you’re interested in Emacs, you should check it out. You don’t have to be a G+ member to read the posts.

Also interesting in that post is a link to A Curious Look at GNU Emacs’s 1000+ Default Keybindings. In it, Lee looks at the default keybindings and finds many strange things. Some of the bindings are historical artifacts and not used anymore. Some are accessed with keys that don’t appear on any modern keyboards. It’s an entertaining post with lots of things I didn’t know before. There’s even a picture of the Symbolics Space Cadet keyboard that had some of those mysterious keys that are no longer extant.

Update: binding → bindings

Posted in General | Tagged | Leave a comment

Safety And The Lisp Read Function

Posted on 2012-01-17 by jcs

For all my blathering about security, here’s a potential exploit in Common Lisp that I never thought about. William Halliburton has a nice explanation of Lisp’s read function and the assoicated read-macro #.. When read sees the sequence #. it will evaluate the next Lisp object that it reads. For example, given the following inputs to read the corresponding object is returned as the result of the read: 

(+ 1 (/ 8 4)) → (+ 1 (/ 8 4))
#.(+ 1 (/ 8 4)) → 3
(+ 1 #.(/ 8 4)) → (+ 1 2)

That’s pretty neat but as Halliburton points out it also poses a security threat because a user can use it to execute arbitrary code in a Lisp program that naively reads data from the user.

Follow the link for a more complete explanation of the #. read-macro, an example of using it for an exploit, and a simple safe-read function that prevents the problem.

If you’d like to play around with read and #. to see how they act and what they return, here’s a function that acts as a REPL without the evaluation that you can use. 

(defun rpl ()
  "A REPL without the evaluate. Type (quit) to exit."
  (princ "rpl> ")
  (let ((obj (read)))
    (prin1 obj)
    (terpri)
    (unless (equal obj '(quit))
      (rpl))))
Posted in Programming | Tagged | 1 Comment