Filtering Lisp-Based Log Files With Emacs Wrap-Up

Yesterday’s post discussed some rudimentary methods of having log files stored as Lisp filter themselves. Today I planned to finish that discussion and remark on how Emacs provides a powerful environment for exploratory programming. The discussion of the 3 challenge questions from yesterday turned out to be longer than I anticipated, so I’ll discuss them today and make my final comments about Emacs and exploratory programming in a subsequent post.

The first question wants us to combine queries with or rather than and. In particular, how would you filter on

level equals SEVERE or line > 30

One way of doing this is to make the definitions

(defun level (s)
  (not (eq s 'SEVERE)))

(defun line (l)
  (<= line 30))

and then use remove-if instead of remove-if-not to filter the records.

The second question asks how you would filter on the method field that is part of the frame sexpr. The easiest way of doing that is to change the definition of frame to return the truth value of the condition. For example, if you wanted to filter on method being ‘main, you could define frame as

(defun frame (class method line)
  (eq method 'main))

The third question shows that we’ve just been fooling around so far and that what we really need is access to all the fields. Our record structure looks like

and the record function can’t see the emessage¹, class, method, and line fields that are underneath the exception node. To get a general solution we need to arrange for record to have access to all the fields.

We start by having the leaf nodes just return their value. Thus, we define

(defalias 'date 'identity)

and similarly for millis, sequence, logger, level, class, thread, emessage, method, and line.

Next, we define the internal nodes exception and frame to return a list of their leaf values:

(defalias 'exception 'list)

and similarly for frame.

With these definitions, record has access to all the fields. To see what record‘s arguments look like, define record as list and execute one of the records. We get

("2005-02-21T18:57:39" 1109041059800 2 nil SEVERE
"java.util.logging.LogManager$RootLogger" log 10 "A very very bad
thing has happened!" ("java.lang.Exception" ("logtest" main 120)))

To get access to the leaves under exception, we need some accessor functions.

(defun exception.emessage ()
  (car exception))

(defun exception.frame ()
  (cadr exception))

(defun exception.frame.class ()
  (car (exception.frame)))

(defun exception.frame.method ()
  (cadr (exception.frame)))

(defun exception.frame.line ()
  (caddr (exception.frame)))

With these definitions we can filter on conditions by writing the appropriate record function. To filter on

(level equals SEVERE or line > 30) and thread equals 10

as the third question asked, we would define record as

(defun record (date millis sequence logger level class method thread emessage exception)
  (and (or (eq level 'SEVERE) (> (exception.frame.line) 30))
       (= thread 10)))

and then evaluate

(remove-if-not 'eval (cdr log-file))

as before.

Note that given some record structure, everything but the record function is fixed so that filtering on different criteria means that only the record function needs to be changed.

UPDATE: would → wouldn’t