I ran across the Open States Website, which provides information on legislative activity for many (and eventually all) of the U.S. states. They have an API that allows you to query information via http and get answers formatted as JSON. I thought it would be fun to play with this in Lisp but I didn’t have a Web client so I asked Google about Lisp Web clients and got pointed to DRAKMA, Edi Weitz’s Common Lisp Web client. DRAKMA provides a simple and easy to use library that allows you to make HTTP requests in Common Lisp.
I loaded it with Quicklisp and was quickly retrieving data for my state. The Open States site is a nice resource if you live in the United States and want to keep an eye on what your state legislators are up to.
While I was playing with DRAKMA it occurred to me that it would have been perfect for investigating the malware problem that I had a while ago. If you followed that sorry tale, you’ll recall that one of my site’s WordPress PHP functions was modified to serve some malicious JavaScript for Windows users. As part of cleaning up the site I needed to verify that the JavaScript was no longer being served. I originally did that with curl but that was a little limited. Using DRAKMA I am able to send a request, pretending I’m a Windows/MSIE user, and check all the JavaScript scripts that come back. Here’s a sample run on the now clean site.
DRAKMA-USER> (ppcre:all-matches-as-strings "<script.*?</script>" (http-request "http://irreal.org/blog/" :user-agent :explorer)) GET /blog/ HTTP/1.1 Host: irreal.org User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727) Accept: */* Connection: close HTTP/1.1 200 OK Date: Sat, 28 Jul 2012 16:00:21 GMT Server: Apache X-Pingback: http://irreal.org/blog/xmlrpc.php X-Powered-By: PHP/5.2.17 Vary: * Connection: close Transfer-Encoding: chunked Content-Type: text/html; charset=UTF-8 ("<script type='text/javascript' src='http://irreal.org/blog/wp-includes/js/jquery/jquery.js?ver=1.7.2'></script>" "<script type='text/javascript' src='http://irreal.org/blog/wp-content/plugins/nucaptcha/res/js/wp-nucaptcha-form.js?ver=3.4.1'></script>")
As you can see, only two scripts are served and they are both legitimate WordPress scripts. I set DRAKMA to print the HTTP headers so that I could verify that the correct USER_AGENT was being sent. This is very nice and since you have the power of Lisp at your disposal, it’s easy to ask any appropriate question about the data that comes back. In the above example, for instance, I used Weitz’s Portable Perl Compatible Regular Expressions (PPCRE) library to pick out any scripts in the output.