Solution to the Two Challenges

The other day, I presented two Elisp coding challenges. I specified Elisp because Irreal readers tend to like Emacs related posts. The down side of that is that several of you worried about bignums and other Emacs limitations. That wasn’t my intent—I just thought the problems were interesting and might even come in handy someday in an interview.

In any event, here are my solutions. All the commenters who gave an answer for problem 1 (Given a list of integers from 1 to n but with one of the integers missing, write an efficient algorithm to find the missing integer.) gave the same O(n)/O(1) algorithm for time/space, namely just sum the integers up and subtract the sum from (n2 + n) / 2 to find the missing number.

The solution to the second problem (Given a list of integers from 1 to n but with one of the integers missing and another repeated, find an efficient algorithm to find the missing and repeated numbers.) is similar except the we sum the numbers and their squares. If we let delta1 be the difference between the sum of the integers and the sum of the integers from 1 to n, and delta2 be the difference between the sum of the squares of the integers and the sum of the squares of the integers from 1 to n, we have:

  • repeatedmissing = delta1 and
  • repeated2missing2 = delta2.

From there, some very easy algebra (the quadratic terms drop out) we get

  • repeated = (delta2 + delta12) / 2 delta1
  • missing = (delta2delta12) / 2 delta1

Now it’s easy to write an algorithm that’s linear in time and constant in space (modulo some increase in size for bignums if we’re using a language that supports them).

(require 'cl)
(defun solve (n repeat missing)
  (let ((numbers (cons repeat
                       (delete missing (loop for i from 1 to n collect i))))
        (sum-1-to-n (/ (* n (1+ n)) 2))
        (sumsq-1-to-n (/ (* n (1+ n) (+ n n 1)) 6))
        (units 0)
        (squares 0))
    (dolist (i numbers)
      (incf units i)
      (incf squares (* i i)))
    (let ((delta1 (- units sum-1-to-n))
          (delta2 (- squares sumsq-1-to-n)))
      (message "repeated number is %d, missing number is %d"
               (/ (+ delta2 (* delta1 delta1)) (* 2 delta1))
               (/ (- delta2 (* delta1 delta1)) (* 2 delta1))))))

I didn’t bother to randomize the list because I don’t make any use of the fact that it’s (almost) in numerical order. Some readers calculated the length of the list but I assumed that n was given as part of the problem. If not, it’s simple to count them up as we sum the numbers and their squares and then calculate sum-1-to-n and sumsq-1-to-n afterwards.

When we run solve, we get the expected answer

ELISP> (solve 100 35 78)
"repeated number is 35, missing number is 78"
ELISP> 
Posted in Programming | Tagged , | 2 Comments

Playing with Common Lisp’s Compiler Macros

Back in April, Robert Smith of Symbo1ics Ideas wrote an excellent post on solving the m-of-n Boolean Circuit problem. The problem is nominally about building a boolean circuit having n inputs that returns TRUE if at least m of the inputs are TRUE. Most of us aren’t electrical engineers, of course, so Smith quickly recasts the problem as building a function that does the same thing.

That’s a pretty easy problem with some obvious solutions but Smith asks how we might go about finding an optimal solution. Optimal in the sense that each input is examined at most once and that a solution is returned as soon as the answer can be definitely determined. That leaves out solutions that just spin through the inputs counting those with a value of TRUE. Smith finds a general formula for the solution and uses that to write a function to make the check.

Then things get interesting. Next, instead of writing a function to make the check he writes a function to generate Lisp code to make the check. With just a little cleverness—you’ll think, “Oh, I would have thought of that”—he has the function generating optimal code. The next step is to take that code and turn it into a compiler macro. Now when the function is called to check if m of the n inputs are TRUE the function will generate direct code if it can and call the iterative function if it can’t.

This is the best description of compiler macros that I’ve come across. I was going to blog about them myself but I couldn’t do better without just copying his post so you should definitely go take a look if you’re a Lisper.

Posted in Programming | Tagged | Leave a comment

Two Elisp Challenges

I ran across a couple of nice interview questions and an interesting story over at Tanya Khovanova’s Math Blog. The two questions are:

  1. Given a list of integers from 1 to n but with one of the integers missing, write an efficient algorithm for finding the missing integer.
  2. Given a list of integers from 1 to n but with one of the integers missing and another integer repeated, write an efficient algorithm for finding the missing and repeated integers.

In both problems the numbers in the list are in no particular order.

So the challenge is to write the two algorithms in Emacs Lisp. The interviewer expected a O(n log n) algorithm for time for the second problem. Can you do better?

Don’t go over to Khnovanova’s blog until you have solved the problems because there are spoilers in the comments. After you solve the problem, though, be sure to read her story about “hiring the smartest people in the world.”

Posted in Programming | Tagged , | 8 Comments

DOCUMENTATION-TEMPLATE

I’m just starting a Lisp project that, among other things, involves fiddling with dates. Working with dates isn’t particularly hard but it’s also not very rewarding. Oddly, Common Lisp doesn’t have any functions to format and manipulate dates. I certainly didn’t want to write a bunch of uninteresting code so I fired up Quicklisp to see what was available. Happily, I found the Simple-Date-Time library that did all the things I wanted it to.

The only problem was that there are a lot of functions but no documentation other than the doc strings in the functions. I thought about writing a quick script to extract them for use as documentation when I thought that this was surely something that someone else had already done. So I fired up Quicklisp again and found Edi Weitz’s DOCUMENTATION-TEMPLATE library. I had Quicklisp retrieve it for me, loaded it at the REPL, typed

(create-template 'simple-date-time :target #P"~/Desktop/dt.html")

and ended up with nicely formatted html documentation that I added to the documentation stack in my browser.

Sadly, it only works with LispWorks, SBCL, and AllegroCL but if you have one of those systems and need some documentation for a library, this is a quick and easy way to generate it. If you want to see what the finished product looks like, here is the documentation for DOCUMENTATION-TEMPLATE generated with DOCUMENTATION-TEMPLATE.

Posted in Programming | Tagged | Leave a comment

Checking The Easy Cases First

Over the weekend, I was amusing myself with this problem from the always entertaining Programming Praxis. The problem is to partition the set {1, 2, ..., 16} into two equally sized subsets such that

  • The sums of the integers in each subset are equal.
  • The sums of the squares of the integers in each subset are equal.
  • The sums of the cubes of the integers in each subset are equal.

That seems straight forward enough. My strategy was to generate combinations of the 16 integers 8 at a time and for each combination, s1:

  1. Calculate the set difference, s2, of {1, 2, ..., 16} with s1.
  2. Check if the sums in S1 and s2 were equal.
  3. Return the answer (s1 s2) if so,
  4. Get the next combination and return to step 1 if not.

That plan worked remarkably wells and gave me the answer in under .02 seconds.

Then I realized that the goal sums were known in advance. For example each sum of integers would have to be one-half the sum of 1 to 16 or 68. Using that fact meant that the number of summings was cut in half and there was no need to calculate a set difference until s1 had the correct sums. That cut the running time in about half. I felt pretty good about that but then I wondered how many correct sums for the integers, the squares, and the cubes there were.

So I adapted the code to count how many combinations summed to the correct amount for each type of sum. There were lots of integer combinations that were correct, just a few combinations of the squares, and exactly 1 of the cubes. Now the code is:

(dolist (c all-combinations)
  (if (= (reduce (lambda (x y) (+ x (* y y y))) c :initial-value 0) 9248)
      (return (list c
                    (set-difference '(1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16)
                                    c)))))

which produced the correct answer in .002 seconds. The point of this post is that it can pay to experiment a bit before you write a lot of code 1.

Footnotes:

1 Of course, this is a bit silly since you have to write most of the code to discover the fact that finding the correct sum of cubes suffices but the point stands: a little experimentation can save a bunch of time.

Posted in Programming | Tagged | 1 Comment

Troubleshooting Emacs Libraries

While trawling around the WikEmacs site today, I came across this page on troubleshooting that explained some commands that I wasn’t familiar with. The first two are helpful if you are trying to search out library problems. For example, if you want to know where your version of org-mode is coming from you could find out with 【Meta+xlocate libraryReturnorgReturn】 and Emacs will tell you where it is loading the library from

Library is file ~/tools/org-mode/lisp/org.elc

The related command list-load-path-shadows will show you if your load-path variable is causing one version of a library to shadow another. For example, when I run it, it tells me that the version of org-mode that came with Emacs is shadowed by the newer version from my tools directory.

Finally, and perhaps most useful, is the find-library command. Suppose you want to look at the code for a library you are using (including the core libraries), let’s say key-chord.el. Merely type 【Meta+xfind-libraryReturnkey-chordReturn】 and Emacs will open a buffer in a new window with the code loaded in it. Very handy.

Posted in General | Tagged | 1 Comment

TPK In Common Lisp

Yesterday I wrote about the Trabb Pardo Knuth algorithm and gave an implementation in Emacs Lisp. Elisp allows an nice implementation but was a bit frustrating because the Elisp interpreter handles overflows internally and never signals an overflow condition. Therefore, I decided to try it in Common Lisp to see how they differ and also to force an overflow.

You can’t get an integer overflow in Common Lisp because of the bignum support but you can with floating point calculations. Here’s my CL implementation of the TPK algorithm. I tried to make it as much like the Elisp version as possible.

(defun tpk ()
  (let ((s (make-array 11)))
    (dotimes (i 11)
      (princ "Enter number: " *query-io*)
      (setf (svref s i) (read *query-io*)))
    (map nil (lambda (n)
               (print (handler-case (expt 2.0 n)
                        (floating-point-overflow () "OVERFLOW"))))
         (reverse s))))

One nice thing about CL is that reverse works on any simple sequence and thus works on vectors. On the other hand, there isn’t a read-number function so I had to use read-line and parse-integer instead. Also, the CL mapc requires lists so I used map, which accepts vectors, instead.

When I ran tpk, I got gratifying results.

CL-USER> (tpk)
Enter number: 1
Enter number: 10
Enter number: 100
Enter number: 200
Enter number: 300
Enter number: 400
Enter number: 500
Enter number: 600
Enter number: 700
Enter number: 800
Enter number: 900

"OVERFLOW" 
"OVERFLOW" 
"OVERFLOW" 
"OVERFLOW" 
"OVERFLOW" 
"OVERFLOW" 
"OVERFLOW" 
"OVERFLOW" 
1.2676506e30 
1024.0 
2.0 
NIL

Update: Pascal Bourguignon pointed out that the specification said to read in numbers not integers so I’ve fixed the code above to use read rather than (parse-integer (read-line *query-io*)).

Posted in Programming | Tagged , , | 2 Comments

The Trabb Pardo Knuth Algorithm in Elisp

The latest Programming Praxis Exercise is interesting. Back in 1973, Luis Trabb Pardo and Don Knuth published an algorithm that was meant to assess the power of a programming language. The algorithm was

Ask for 11 numbers to be read into a sequence S
Reverse S
For each number in S
  Call some function to do a mathematical operation on the number
  If the result overflows
    Alert the user
  else
    Print the result

That seems simple enough but notice how it involves arrays (lists would be more natural in Lisp but the intent was to use arrays), indexing, I/O, iteration, general functions, mathematical functions, and error detection. Be implementing the algorithm in a language you get a good feel for how the language handles common programming tasks.

Naturally, I had to try it out in Lisp. I used Emacs Lisp as a sort of minimal Lisp but still got a very nice and concise implementation.

(require 'cl)
(let ((s (make-vector 11 0)))
  (dotimes (i 11)
    (aset s i (read-number "Enter a number: ")))
  (coerce (reverse (coerce s 'list)) 'vector)
  (mapc (lambda (n)
          (print (condition-case nil (expt 2 n) ((overflow-error) "OVERFLOW"))))
        s))

If I had used a list instead of an array, I wouldn’t have needed the two calls to coerce but I would have needed two reverses if I filled the list in the most natural way.

The condition-case is like try/except from other languages. It performs the exponentiation and returns the result unless an overflow error occurs in which case it returns OVERFLOW. Whatever is returned is then printed. The only problem I had was getting an overflow. Generating an overflow is hard in most Lisps because of their bignum facilities. Emacs Lisp doesn’t use bignums but it does handle overflows in the interpreter. A sexpr like (expt 10 1000) just returns 0. If you use (expt 10.0 1000) it returns 1.0e+INF. Division by zero does generate an error but its an arith-error not an overflow-error. If anyone knows how to generate an overflow in Elisp, leave a comment.

Posted in Programming | Tagged , | 4 Comments

The WikEmacs Elisp Cookbook

I stopped by WikEmacs today to see how the site was progressing. It looks pretty nice and has obviously seen some hard work by its contributors. Being me, I immediately went to the Emacs Lisp Cookbook to see how it looked. It’s pretty much the same as the EmacsWiki Elisp Cookbook but a little better formatted and with some of the material rearranged. So while there isn’t much new material it is nice to see that it’s migrated to WikEmacs.

I don’t take a stand on whether one site is better than the other; I’m just happy to see more information about Emacs collected together wherever folks feel most comfortable contributing. If you haven’t checked WikEmacs out lately, drop by and see what you think.

Posted in Programming | Leave a comment

And So It Begins

Tor/Forge, part of the Macmillan empire announced that they will start shipping their ebooks without DRM starting in July. This is the beginning of the DRM death spiral predicted by Charlie Stross (although he has tempered that prediction a bit in his analysis of Tor’s announcement).

John Scalzi weighns in with his own thoughts and an announcement that his forthcoming novel, Redshirts, will be DRM-free even though it will be released in June. Like Stross, he views this as a positive move.

I am hopeful that this action will trigger the collapse of DRM in the publishing industry. Once the publishers see that the earth continues to orbit the sun even without DRM, I expect that they will all jump on the bandwagon.

Posted in General | Tagged | Leave a comment