Scheme and Common Lisp

I’ve been using Scheme and Common Lisp almost exclusively for over 10 years. During that time, I tended to favor Scheme because I liked its clean design and simplicity, the named let, continuations, and the simplicity that being a Lisp-1 brought. As I posted recently, Guile Scheme is broken on OS X Lion. It’s still broken and as a result I have been using Common Lisp exclusively.

That has forced me to understand Common Lisp better and to become a more proficient Lisp programmer. At this point, I’m ready to say goodbye to Scheme. Common Lisp doesn’t have as simple a design but it is much more powerful. It’s easy to implement the named let functionality with a macro (see LoL for an implementation) and as Doug Hoyte argues in Let Over Lambda, Lisp-2 is arguably better than Lisp-1. I’m not yet convinced of that last statement but using CL exclusively for the last month has made the difference fade into the background. That leaves only continuations. They’re certainly powerful but CL provides a way of doing everything I used them for.

One huge advantage of Common Lisp is that it’s standardized in a way that Scheme isn’t. I recently changed from Clozure CL to SBCL and I haven’t had to make any code changes. It’s very hard to do the same thing is Scheme. When I changed from Dr. Scheme (now Dr. Racket) to Guile I had to rewrite a lot of code. That carries over to libraries. It was pretty hard to find libraries that worked with any Scheme and although there are certainly CL libraries that use non standard features of a certain implementation and are not as a result portable, most libraries work just fine with any Lisp. Notable exceptions are things, like networking, that are not part of the standard. But even there, Zack Beane shows a very nice way of handling the differences with his Quicklisp implementation.

Finally, there’s a subtle difference in the feel of the two languages. CL seems to give me more power to solve problems. Thus, although I was a little annoyed that Guile won’t build under Lion (I know, it’s free and the code is there to fix), in retrospect, I think it was actually a good thing. It has made me reconnect strongly with my first Lisp and that’s working out very well for me.

Posted in Programming | Tagged , | 4 Comments

Using Quicklisp

Back in March I wrote that I had loaded and starting using Zach Beane’s Quicklisp. Since then I have used it with several small projects and I really like it. It does two things for me:

  • Makes it super easy to download libraries (see this post, for example) and keep them up to date.
  • Manage my own projects.

That second item is, in many ways, even nicer than the ease with which you can retrieve libraries. Quicklisp can also load your local projects, so it’s easy to just do a

(ql:quickload "some-local-project")

to load your project into Lisp so that you can continue working on it. If your project is a library, Quicklisp will automatically load it for other projects that depend on it (via the .asd file).

Today, I was trawling around on Beane’s site, Xach.com, and came across an excellent article he wrote about Making a small Lisp project with quickproject and Quicklisp. If you’re a Lisper and haven’t read this, you should head on over right now to see how a master does things.

One of the interesting things I learned from the article was quickproject. If you’ve ever put together a system with ASDF, you know that the process can be a little fiddly. You need to build the package.lisp file, the project.asd file, and your actual code files. Quickproject does much of the work for you. It will build default package.lisp and project.asd files based on the information you give it. It will even make an empty project.lisp file so that you open it in your editor and get right to work.

Naturally, quickproject is available with Quicklisp so I loaded onto my system. I haven’t had a chance to use it yet but I plan to start with my next project.

Posted in Programming | Leave a comment

A Message From The EFF

Posted in General | Leave a comment

Solution to the Two Challenges

The other day, I presented two Elisp coding challenges. I specified Elisp because Irreal readers tend to like Emacs related posts. The down side of that is that several of you worried about bignums and other Emacs limitations. That wasn’t my intent—I just thought the problems were interesting and might even come in handy someday in an interview.

In any event, here are my solutions. All the commenters who gave an answer for problem 1 (Given a list of integers from 1 to n but with one of the integers missing, write an efficient algorithm to find the missing integer.) gave the same O(n)/O(1) algorithm for time/space, namely just sum the integers up and subtract the sum from (n2 + n) / 2 to find the missing number.

The solution to the second problem (Given a list of integers from 1 to n but with one of the integers missing and another repeated, find an efficient algorithm to find the missing and repeated numbers.) is similar except the we sum the numbers and their squares. If we let delta1 be the difference between the sum of the integers and the sum of the integers from 1 to n, and delta2 be the difference between the sum of the squares of the integers and the sum of the squares of the integers from 1 to n, we have:

  • repeatedmissing = delta1 and
  • repeated2missing2 = delta2.

From there, some very easy algebra (the quadratic terms drop out) we get

  • repeated = (delta2 + delta12) / 2 delta1
  • missing = (delta2delta12) / 2 delta1

Now it’s easy to write an algorithm that’s linear in time and constant in space (modulo some increase in size for bignums if we’re using a language that supports them).

(require 'cl)
(defun solve (n repeat missing)
  (let ((numbers (cons repeat
                       (delete missing (loop for i from 1 to n collect i))))
        (sum-1-to-n (/ (* n (1+ n)) 2))
        (sumsq-1-to-n (/ (* n (1+ n) (+ n n 1)) 6))
        (units 0)
        (squares 0))
    (dolist (i numbers)
      (incf units i)
      (incf squares (* i i)))
    (let ((delta1 (- units sum-1-to-n))
          (delta2 (- squares sumsq-1-to-n)))
      (message "repeated number is %d, missing number is %d"
               (/ (+ delta2 (* delta1 delta1)) (* 2 delta1))
               (/ (- delta2 (* delta1 delta1)) (* 2 delta1))))))

I didn’t bother to randomize the list because I don’t make any use of the fact that it’s (almost) in numerical order. Some readers calculated the length of the list but I assumed that n was given as part of the problem. If not, it’s simple to count them up as we sum the numbers and their squares and then calculate sum-1-to-n and sumsq-1-to-n afterwards.

When we run solve, we get the expected answer

ELISP> (solve 100 35 78)
"repeated number is 35, missing number is 78"
ELISP> 
Posted in Programming | Tagged , | 2 Comments

Playing with Common Lisp’s Compiler Macros

Back in April, Robert Smith of Symbo1ics Ideas wrote an excellent post on solving the m-of-n Boolean Circuit problem. The problem is nominally about building a boolean circuit having n inputs that returns TRUE if at least m of the inputs are TRUE. Most of us aren’t electrical engineers, of course, so Smith quickly recasts the problem as building a function that does the same thing.

That’s a pretty easy problem with some obvious solutions but Smith asks how we might go about finding an optimal solution. Optimal in the sense that each input is examined at most once and that a solution is returned as soon as the answer can be definitely determined. That leaves out solutions that just spin through the inputs counting those with a value of TRUE. Smith finds a general formula for the solution and uses that to write a function to make the check.

Then things get interesting. Next, instead of writing a function to make the check he writes a function to generate Lisp code to make the check. With just a little cleverness—you’ll think, “Oh, I would have thought of that”—he has the function generating optimal code. The next step is to take that code and turn it into a compiler macro. Now when the function is called to check if m of the n inputs are TRUE the function will generate direct code if it can and call the iterative function if it can’t.

This is the best description of compiler macros that I’ve come across. I was going to blog about them myself but I couldn’t do better without just copying his post so you should definitely go take a look if you’re a Lisper.

Posted in Programming | Tagged | Leave a comment

Two Elisp Challenges

I ran across a couple of nice interview questions and an interesting story over at Tanya Khovanova’s Math Blog. The two questions are:

  1. Given a list of integers from 1 to n but with one of the integers missing, write an efficient algorithm for finding the missing integer.
  2. Given a list of integers from 1 to n but with one of the integers missing and another integer repeated, write an efficient algorithm for finding the missing and repeated integers.

In both problems the numbers in the list are in no particular order.

So the challenge is to write the two algorithms in Emacs Lisp. The interviewer expected a O(n log n) algorithm for time for the second problem. Can you do better?

Don’t go over to Khnovanova’s blog until you have solved the problems because there are spoilers in the comments. After you solve the problem, though, be sure to read her story about “hiring the smartest people in the world.”

Posted in Programming | Tagged , | 8 Comments

DOCUMENTATION-TEMPLATE

I’m just starting a Lisp project that, among other things, involves fiddling with dates. Working with dates isn’t particularly hard but it’s also not very rewarding. Oddly, Common Lisp doesn’t have any functions to format and manipulate dates. I certainly didn’t want to write a bunch of uninteresting code so I fired up Quicklisp to see what was available. Happily, I found the Simple-Date-Time library that did all the things I wanted it to.

The only problem was that there are a lot of functions but no documentation other than the doc strings in the functions. I thought about writing a quick script to extract them for use as documentation when I thought that this was surely something that someone else had already done. So I fired up Quicklisp again and found Edi Weitz’s DOCUMENTATION-TEMPLATE library. I had Quicklisp retrieve it for me, loaded it at the REPL, typed

(create-template 'simple-date-time :target #P"~/Desktop/dt.html")

and ended up with nicely formatted html documentation that I added to the documentation stack in my browser.

Sadly, it only works with LispWorks, SBCL, and AllegroCL but if you have one of those systems and need some documentation for a library, this is a quick and easy way to generate it. If you want to see what the finished product looks like, here is the documentation for DOCUMENTATION-TEMPLATE generated with DOCUMENTATION-TEMPLATE.

Posted in Programming | Tagged | Leave a comment

Checking The Easy Cases First

Over the weekend, I was amusing myself with this problem from the always entertaining Programming Praxis. The problem is to partition the set {1, 2, ..., 16} into two equally sized subsets such that

  • The sums of the integers in each subset are equal.
  • The sums of the squares of the integers in each subset are equal.
  • The sums of the cubes of the integers in each subset are equal.

That seems straight forward enough. My strategy was to generate combinations of the 16 integers 8 at a time and for each combination, s1:

  1. Calculate the set difference, s2, of {1, 2, ..., 16} with s1.
  2. Check if the sums in S1 and s2 were equal.
  3. Return the answer (s1 s2) if so,
  4. Get the next combination and return to step 1 if not.

That plan worked remarkably wells and gave me the answer in under .02 seconds.

Then I realized that the goal sums were known in advance. For example each sum of integers would have to be one-half the sum of 1 to 16 or 68. Using that fact meant that the number of summings was cut in half and there was no need to calculate a set difference until s1 had the correct sums. That cut the running time in about half. I felt pretty good about that but then I wondered how many correct sums for the integers, the squares, and the cubes there were.

So I adapted the code to count how many combinations summed to the correct amount for each type of sum. There were lots of integer combinations that were correct, just a few combinations of the squares, and exactly 1 of the cubes. Now the code is:

(dolist (c all-combinations)
  (if (= (reduce (lambda (x y) (+ x (* y y y))) c :initial-value 0) 9248)
      (return (list c
                    (set-difference '(1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16)
                                    c)))))

which produced the correct answer in .002 seconds. The point of this post is that it can pay to experiment a bit before you write a lot of code 1.

Footnotes:

1 Of course, this is a bit silly since you have to write most of the code to discover the fact that finding the correct sum of cubes suffices but the point stands: a little experimentation can save a bunch of time.

Posted in Programming | Tagged | 1 Comment

Troubleshooting Emacs Libraries

While trawling around the WikEmacs site today, I came across this page on troubleshooting that explained some commands that I wasn’t familiar with. The first two are helpful if you are trying to search out library problems. For example, if you want to know where your version of org-mode is coming from you could find out with 【Meta+xlocate libraryReturnorgReturn】 and Emacs will tell you where it is loading the library from

Library is file ~/tools/org-mode/lisp/org.elc

The related command list-load-path-shadows will show you if your load-path variable is causing one version of a library to shadow another. For example, when I run it, it tells me that the version of org-mode that came with Emacs is shadowed by the newer version from my tools directory.

Finally, and perhaps most useful, is the find-library command. Suppose you want to look at the code for a library you are using (including the core libraries), let’s say key-chord.el. Merely type 【Meta+xfind-libraryReturnkey-chordReturn】 and Emacs will open a buffer in a new window with the code loaded in it. Very handy.

Posted in General | Tagged | 1 Comment

TPK In Common Lisp

Yesterday I wrote about the Trabb Pardo Knuth algorithm and gave an implementation in Emacs Lisp. Elisp allows an nice implementation but was a bit frustrating because the Elisp interpreter handles overflows internally and never signals an overflow condition. Therefore, I decided to try it in Common Lisp to see how they differ and also to force an overflow.

You can’t get an integer overflow in Common Lisp because of the bignum support but you can with floating point calculations. Here’s my CL implementation of the TPK algorithm. I tried to make it as much like the Elisp version as possible.

(defun tpk ()
  (let ((s (make-array 11)))
    (dotimes (i 11)
      (princ "Enter number: " *query-io*)
      (setf (svref s i) (read *query-io*)))
    (map nil (lambda (n)
               (print (handler-case (expt 2.0 n)
                        (floating-point-overflow () "OVERFLOW"))))
         (reverse s))))

One nice thing about CL is that reverse works on any simple sequence and thus works on vectors. On the other hand, there isn’t a read-number function so I had to use read-line and parse-integer instead. Also, the CL mapc requires lists so I used map, which accepts vectors, instead.

When I ran tpk, I got gratifying results.

CL-USER> (tpk)
Enter number: 1
Enter number: 10
Enter number: 100
Enter number: 200
Enter number: 300
Enter number: 400
Enter number: 500
Enter number: 600
Enter number: 700
Enter number: 800
Enter number: 900

"OVERFLOW" 
"OVERFLOW" 
"OVERFLOW" 
"OVERFLOW" 
"OVERFLOW" 
"OVERFLOW" 
"OVERFLOW" 
"OVERFLOW" 
1.2676506e30 
1024.0 
2.0 
NIL

Update: Pascal Bourguignon pointed out that the specification said to read in numbers not integers so I’ve fixed the code above to use read rather than (parse-integer (read-line *query-io*)).

Posted in Programming | Tagged , , | 2 Comments