Irreal

This is sort of a note to myself. I sometimes need to know the character designation of a syntax class. Most often this is for use with the \sC syntax in a regexp. For example

(looking-at "^\\s-*$\\|\\s-*;")

asks if we’re looking at a line containing only whitespace or a line with possible leading whitespace and then a ;. (It’s used to recognize lines in a Scheme program file with no executable code.)

The problem is I always have a hard time finding the Syntax Class Table in the Elisp manual. Xah Lee has helpfully provided a bunch of pages like HTML/XML Entities List, Unicode Space Symbols, and a Regexp Cheat Sheet that I’ve bookmarked for easy reference but nothing for the syntax table. Therefore, I’m making this short table mostly so I can bookmark it.

The characters in a given syntax class usually depend on the current major mode so the table doesn’t give the full story. For that you need to look at the Syntax Class Table node of the Elisp manual.

Update: Designated hyphen and underscore explicitly.

Sacha Chua introduced me to regexp-opt, an Emacs function that I hadn’t seen before. The idea is that regexp-opt will take a list of strings and return a more or less optimal regexp to recognize any of those strings.

Because it does analysis and restructuring of the list of strings, regexp-opt will generally produce a more efficient regexp than the equivalent

(mapconcat 'regexp-quote strings "\\|")

In addition to the list of strings, regexp-opt takes an optional second argument that if not nil will surround the regexp with grouping parentheses. If the second argument is 'words, then regexp-opt also surrounds the regexp with \\< and \\>.

Here’s an example from the source code for regexp-opt that shows how it optimizes a regexp.

(let ((strings '("cond" "if" "when" "unless" "while"
                    "let" "let*" "progn" "prog1" "prog2"
                    "save-restriction" "save-excursion" "save-window-excursion"
                    "save-current-buffer" "save-match-data"
                    "catch" "throw" "unwind-protect" "condition-case")))
  (concat "(" (regexp-opt strings t) "\\>"))
 => "(\\(c\\(atch\\|ond\\(ition-case\\)?\\)\\|if\\|let\\*?\\|prog[12n]\\|save-\\(current-buffer\\|excursion\\|match-data\\|restriction\\|window-excursion\\)\\|throw\\|un\\(less\\|wind-protect\\)\\|wh\\(en\\|ile\\)\\)\\>"

This regexp takes approximately two-thirds the amount of time that the equivalent mapconcat regexp does.

Simon Marshall, the code’s author, notes that the package was written to produce efficient regexps, not to produce regexps efficiently and therefore you should avoid using it in-line too much unless you use eval-when-compile. See the commentary in regexp-opt.el for more on this. You can, of course, easily get to the code from the function description by clicking on the file name.

Update: Fixed mapconcat example.

A few days ago, I posed an Elisp programming challenge that asked you to make the record

(record
  (date "2005-02-21T18:57:39")
  (millis 1109041059800)
  (sequence 1)
  (logger nil)
  (level 'SEVERE)
  (class "java.util.logging.LogManager$RootLogger")
  (method 'log)
  (thread 10)
  (emessage "A very very bad thing has happened!")
  (exception
    (emessage "java.lang.Exception")
    (frame
      (class "logtest")
      (method 'main)
      (line 30))))

executable in such a way that it transforms itself into the equivalent XML record when executed.

On the day I posed the challenge, I woke up thinking that making the record transform itself into XML was the proper way to turn it into XML rather than the way I described in Converting S-Expression To XML In Emacs. It seemed like a direct application of the ideas in my More Fun With Log Files Stored As Lisp post and I decided to program it just for fun with no idea of turning it into a post.

A couple of hours later I was still flailing around so maybe the application wasn’t as direct as I thought. Actually, the idea is the same but because you’re not transforming it back into Lisp it’s a little trickier.

In any event, here’s how I solved the problem. First define all the tags except record to be functions of the form

(defun date (&rest args) (toxml 'date args))

As I explained in the Exploratory Programming In Emacs post that involved little more than copying the list of tags and a keyboard macro. The toxml function is

(defun toxml (tag args)
  (with-output-to-string
    (princ (format "\n<%s>" tag))
    (mapc 'princ args)
    (princ (format "</%s>" tag))))

The idea is that each sexpr returns a string of the arguments wrapped in <tag>…</tag> where tag is the symbol in the function slot of the sexpr. That is, (date "2005-02-21T18:57:39") is transformed to "<date>2005-02-21T18:57:39</date>". If we temporarily alias record to list and executed the record sexpr, we get

("
<date>2005-02-21T18:57:39</date>" "
<millis>1109041059800</millis>" "
<sequence>1</sequence>" "
<logger>nil</logger>" "
<level>SEVERE</level>" "
<class>java.util.logging.LogManager$RootLogger</class>" "
<method>log</method>" "
<thread>10</thread>" "
<emessage>A very very bad thing has happened!</emessage>" "
<exception>
<emessage>java.lang.Exception</emessage>
<frame>
<class>logtest</class>
<method>main</method>
<line>30</line></frame></exception>")

which shows what the record function will see when it gets called. Note that each string begins with a newline so the ” ” at the end of each line is a close quote followed by an open quote.

Finally, we need to define a record function to stitch the strings together and get rid of that annoying nil in <logger>nil</logger>.

(defun record (&rest args)
  (princ "<record>")
  (princ (replace-regexp-in-string ">nil<" "><" (apply 'concat args)))
  (princ "</record>"))

The princ on the third line gets rid of the quote marks that the concat places around the final result of the apply.

Now when we execute the record we get

<record>
<date>2005-02-21T18:57:39</date>
<millis>1109041059800</millis>
<sequence>1</sequence>
<logger></logger>
<level>SEVERE</level>
<class>java.util.logging.LogManager$RootLogger</class>
<method>log</method>
<thread>10</thread>
<emessage>A very very bad thing has happened!</emessage>
<exception>
<emessage>java.lang.Exception</emessage>
<frame>
<class>logtest</class>
<method>main</method>
<line>30</line></frame></exception></record>

Update: Let me mention again that the inspiration for this series of posts and the sample record sexpr are from Steve Yegge’s The Emacs Problem. Be sure to give it a read if you haven’t already.

In Converting S-Expressions To XML In Emacs, I showed how to take a log record expressed as Lisp and turn it into the equivalent XML. In More Fun With Log Files Stored As Lisp, I showed how to take that same log record and make it executable so that it transformed one of the fields when the record was executed.

Here is a challenge for Elisp programmers: take the record

(record
  (date "2005-02-21T18:57:39")
  (millis 1109041059800)
  (sequence 1)
  (logger nil)
  (level 'SEVERE)
  (class "java.util.logging.LogManager$RootLogger")
  (method 'log)
  (thread 10)
  (emessage "A very very bad thing has happened!")
  (exception
    (emessage "java.lang.Exception")
    (frame
      (class "logtest")
      (method 'main)
      (line 30))))

and turn it into XML like this:

<record>
<date>2005-02-21T18:57:39</date>
<millis>1109041059800</millis>
<sequence>1</sequence>
<logger></logger>
<level>SEVERE</level>
<class>java.util.logging.LogManager$RootLogger</class>
<method>log</method>
<thread>10</thread>
<emessage>A very very bad thing has happened!</emessage>
<exception>
<emessage>java.lang.Exception</emessage>
<frame>
<class>logtest</class>
<method>main</method>
<line>30</line></frame></exception></record>

Do this by making the Lisp record executable so that it transforms itself into the XML when executed. Your XML doesn’t have to be formatted exactly like mine and it’s sufficient to have it print out in the echo area or the *Messages* buffer.

I’ll give my solution in a few days so that you have time to work on it. If you get stuck, the wrap-up post has some techniques that may help.

Yesterday, I mentioned that Xah Lee had a short post on dramatically differing run times for processing several files with find-file versus processing the same files using with-temp-buffer. His post was fairly short and didn’t have a lot of details. Happily, he’s published an extended post that shows the code and detailed performance of the two approaches.

This is really interesting and if you write Elisp that looks at lots of files at once, you should check out his post.

Xah Lee has a short post on find-file vs. with-temp-buffer for loading a large number of files. The results were sort of surprising, at least to me, so you should take a look if you need to load several files in Emacs. His post was basically a little benchmark and I vaguely wondered how he did the timing. Scheme and Common Lisp have functions for this but I had never seen anything for Elisp.

Just now, while looking at something completely unrelated I came across the benchmark-run command. It does just exactly what its name suggests: if you want to time some Elisp code you just wrap it with benchmark-run and go. It will report the elapsed time for execution, the number of garbage collections that ran, and the time spent in the garbage collector.

For example, suppose we want to run some-function on several files that are marked in a Dired buffer. We could do something like

(benchmark-run 1 (mapc 'some-function (dired-get-marked-files)))

to time how long it takes. The first argument (1) says to run the benchmark 1 time. You can set that to a larger number if you want to time several runs to get a better idea of average performance.

That’s a pretty nifty thing to know and I don’t recall ever seeing it anywhere before. That’s one of the nice things about Emacs: there’s always something new to learn.

Update: Lisp → Common Lisp