Using set-mark-command To Remember Locations

A couple of days ago I wrote a note to myself post to help me remember the align-regexp functionality. Today’s post is another note to myself. A recent post by Xah Lee reminded me of some Emacs functionality that I am always forgetting. Every time I see the set-mark-command function described I think, “Boy, that’s really useful, I need to remember it.” Then I promptly forget about it again.

The set-mark-command (bound to 【Ctrl+Space】) sets the mark at the point’s position. The normal way of invoking it to remember locations is to type 【Ctrl+Space Ctrl+Space】. The first 【Ctrl+Space】 sets the mark and the second deactivates it so that no region is highlighted. Emacs also pushes the mark onto the buffer’s mark ring and onto the global mark ring.

This is useful because we can use the marks recorded on the mark rings to return to a remembered location. If we want to return to a previous mark in the same buffer, we type 【Ctrl+u Ctrl+Space】 We can move back several locations by repeating the 【Ctrl+u Ctrl+Space】.

If we want to return to a mark which may be in another buffer we can type 【Ctrl+x Ctrl+Space】. This can also be repeated to move back several marks.

This may seem confusing but it boils down to just three key sequences

  • Ctrl+Space Ctrl+Space】 to push the current position onto the mark rings.
  • Ctrl+u Ctrl+Space】 to return to a previous position in the same buffer.
  • Ctrl+x Ctrl+Space】 to return to the previous position, which may or may not be in the same buffer.
Posted in General | Tagged | 6 Comments

More On Borders’ Closing

David Hill over at Singularity Hub has an interesting take on the closing of Borders. While agreeing that Borders’ downfall is a shame, Hill argues that it will actually be a huge win for authors and readers. It will benefit authors, he says, by accelerating the changes in the publishing industry that are putting the authors in control. Those same changes will hasten the move to digital books with all the advantages that brings to readers.

Hill’s post has a link to a post by Alan Rinzier, an industry insider, that is also interesting. Rinzier confirms the trend of greater author control and says, basically, that the publishing industry has no idea what it’s doing and is coming to realize that. As evidence for this, he reports that 80–90 percent of all published books lose money.

Rinzier says that while it’s as hard as ever to write a good book, authors who can now have the opportunity to self publish and control their own destiny. As I suggested in my Future of Books post, the publishing industry is undergoing great change but appears to remain in denial. If they wish to avoid Borders’ fate, they had better get a clue. They still provide a valuable service but they are no longer the gatekeepers that they once were. If their tactics to defend their business models become too annoying, authors and readers will simply route around them and they will learn first hand the meaning of disintermediation. As I also said in that post, I hope that doesn’t happen but I see little reason to be sanguine about their prospects.

Posted in General | Leave a comment

Aligning Text With Regular Expressions in Emacs

This is really a note to myself. A while ago I ran across this post over at RAW SYNTAX about the Emacs command align-regexp. I don’t require this functionality very often but when I do, align-regexp is just what I need. Unfortunately, I keep forgetting about it so I’m writing this post in the hopes that it will help me remember (or at least give me a place to find the name of the function)

The simplest, and probably most common, use case is to align on a particular character. The canonical example is a telephone number list. We just mark the region to align, type 【Meta+xalign-regexp, and then ( when prompted for the regular expression

# Original list

Joe (555) 111-2222
Mary  (555) 222-1212
Alexandra 123-4567
Francis (555) 333-2121

# After align-regexp

Joe     (555) 111-2222
Mary    (555) 222-1212
Alexandra 123-4567
Francis (555) 333-2121

That’s not quite what we want, though, because Alexandra’s number doesn’t have an area code so there is no open parenthesis to align on. To fix this, we have to use the more complicated version of align-regexp by giving the invocation a prefix arg: 【Ctrl+u Meta+xalign-regexp. This time we specify .+?\( +\).+ as the regular expression, choose 1 as the group to modify, 1 as the amount of spacing, and no as to whether we should repeat the rule throughout the line. The result is:

Joe       (555) 111-2222
Mary      (555) 222-1212
Alexandra 123-456
Francis   (555) 333-2121

That’s better but what we really want is to align on the -. If we do that directly using the simple case, we get

Joe (555) 111     -2222
Mary  (555) 222   -1212
Alexandra 123     -4567
Francis (555) 333 -2121

which is definitely not what we want. Instead we use the regular expression .+? +\(.*\) and specify -1 as the group to modify or justify. That gives us

Joe        (555) 111-2222
Mary       (555) 222-1212
Alexandra        123-4567
Francis    (555) 333-2121

which is just what we want.

Update: lower case the key sequences.

Update 2: prefix tag → prefix arg

Posted in General | Tagged | 2 Comments

Traveling Without A Computer

Heh heh. OK, that was a teaser title. Of course, I would never go anywhere without a computer; my days of being ABEND are long over. My normal routine is to take my MacBook Pro and associated equipment with me in a backpack. The MacBook essentially mirrors my main iMac machine so I can do anything on the road that I can do at home. Still, flying with the laptop is a hassle. I use the excellent SwissGear ScanSmart Backpack so at least I don’t have to take the computer out of the backpack but it’s still one more thing to deal with at the TSA checkpoints.

Last week, I was in New York visiting family. I decided to try leaving the MacBook at home and depending on my iPad instead. Naturally that precluded any development or serious writing but this was a family visit so, realistically, I wouldn’t be doing much of either. I could have tried depending on my iPhone and avoiding large computers altogether but I wanted to keep up with blogs and news and it’s just too painful to read that sort of thing on the iPhone’s small screen.

Traveling with the iPad is definitely easier. I have a small case for it and the only support equipment it requires is the charging cable. I always travel with the wonderful Scottevest Travel Vest which, with its 22 pockets, has plenty of room for the cable, my iPhone, keys, and all the other stuff I would normally throw in the backpack. When I got to the checkpoint, my shoes and iPad went into a bin and I put the travel vest on top. Just one bin and the iPad is effectively hidden from anyone tempted to get a free computer as they come out of the X-ray machine.

As I’ve mentioned before, I keep up with my feeds with Reeder, which has a very nice and easy to use iPad implementation so it was easy staying current with blogs and news. The iPad has a nice email client that I use for my main email inbox. Mail from mailing lists and things like that go to a gmail account and gmail on the iPad is so good that I prefer to read gmail on the iPad even when I’m home.

The only other thing I needed to do was to manage my blog. I queued up a couple of posts so I didn’t have to worry about writing any entries while I was on the road (I do all that in Emacs anyway so the iPad is not a place to write posts). I was able to moderate forum spam and respond to legitimate comments easily.

Touch typing just isn’t going to happen on the iPad, but I can do a bit better than hunt and peck on the virtual keyboard so responding to emails and blog comments was relatively painless. I suppose I could get a portable keyboard to use with the iPad but, honestly, I don’t feel the need. For the little amount of typing I do on it, the virtual keyboard is fine.

All in all the experiment was a success. Traveling with the iPad was easier than bringing along the MacBook. The few chores I needed to perform were easily handled on the iPad so I didn’t miss having the MacBook. And as a bonus, I didn’t have to carry a book to read on the plane—I just fired up iBooks and enjoyed an ebook.

Update: Over at IT World Brian Proffitt has a post on using (only) an iPad for 7 days.

Posted in General | Tagged | Leave a comment

Xah’s Challenge (Part 2)

Yesterday, I wrote about Xah Lee’s programming challenge and gave a solution similar to Lee’s own. Today I will give another, possibly simpler, solution. We start with some constants and a couple of helper functions:

(defconst brackets '((?( . ?))
                     (?{ . ?})
                     (?[ . ?])
                     (?“ . ?”)
                     (?‹ . ?›)
                     (?« . ?»)
                     (?【 . ?】)
                     (?〈 . ?〉)
                     (?《 . ?》)
                     (?「 . ?」)
                     (?『 . ?』)))

(defconst not-brackets "^(){}[]“”‹›«»【】〈〉《》「」『』")

(defvar filen nil)

(defun brkt-closes-p (brkt obrkt)
  "Predicate to test if brkt closes obrkt."
  (let ((bpair (assoc obrkt brackets)))
    (and bpair (char-equal brkt (cdr bpair)))))

(defun brkt-open-p (brkt)
  "Predicate to test if brkt is an opening bracket."
  (assoc brkt brackets))

As before, we have a list of the beginning and ending brackets but this time they are given as characters rather than strings. The not-brackets constant will be used in a call to skip-chars-forward. The ^ at the beginning of the string specifies all characters except the brackets.

The brkt-closes-p function is a predicate that tests whether its first argument is the closing bracket for the second argument. It just uses the brackets constant as an alist.

The brkt-open-p function is even simpler. It’s a predicate that tests whether its first argument is an opening bracket. That will be true if it’s found in a key position of the brackets alist.

As in the first solution, check-file-for-mismatches gets called to check a single file but all the work is done in brkt-match. The check-file-for-mismatches function merely sets up the temporary buffer for the file and calls brkt-match.

(defun check-file-for-mismatches (fpath)
  "Check a file for mismatched brackets."
  (let ((fb (get-buffer-create "*Temp*")))
    (set-buffer fb)
    (insert-file-contents fpath nil nil nil t)
    (goto-char 1)
    (setq filen fpath)
    (brkt-match ?\f)
    (kill-buffer fb)))

The real checking is done in brkt-match:

 1:  (defun brkt-match (obrkt)
 2:    "Match OBRKT with its closing bracket."
 3:    (let ((open-pos (1- (point))))
 4:      (catch 'exit
 5:        (while t
 6:          (skip-chars-forward not-brackets)
 7:          (if (not (eobp))
 8:              (forward-char))
 9:          (cond
10:           ((eobp) (if (not (char-equal ?\f obrkt))
11:                       (print (format "Unclosed %c in %s at %d"
12:                                      obrkt filen open-pos)))
13:            (throw 'exit t))
14:           ((brkt-closes-p (char-before) obrkt) (throw 'exit t))
15:           ((brkt-open-p (char-before)) (brkt-match (char-before)))
16:           (t (print (format "Mismatch in %s at %d" filen (1- (point))))))))))
17:  

The idea is that brkt-match is passed an opening bracket and searches for the closing bracket (the ?\f passed in by check-file-for-mismatches is a special marker indicating the beginning of file—it matches end-of-buffer). Rather than locate the brackets with a regular expression, we find them by skipping over everything else with skip-chars-forward on line 6.

The processing is done in the while loop on lines 5–16. The catch on line 4 is so that we can exit the while if we match the input or reach end-of-buffer. The cond on lines 9–16 makes the checks:

  1. Lines 10–13
    If we are at end-of-buffer and the input was not ?\f then output an error message about an unclosed bracket. In any case, exit the loop and therefore the function.
  2. Line 14
    If the current character closes the input character, exit the loop and function.
  3. Line 15
    If this is an opening bracket, recursively call brkt-match to try to match it.
  4. Line 16
    If the other tests fail then this is a mismatched bracket. Output an error message and keep looking for the closing bracket matching the input.

We can use the exact same do-files from the first solution to drive check-file-for-mismatches so I won’t repeat it here.

This solution appears radically different from the first but it isn’t really; only the details differ. We use characters instead of strings, skip-chars-forward instead of regular expressions, and recursion instead of an explicit stack but the strategy is the same:

  • Search for the match to an opening bracket
  • If we find another open bracket, save what we’ve done so far and start over with the new opening bracket
  • When we match an opening bracket, pick up where we left off.

It’s an interesting question as to which method is faster. I haven’t tested that aspect but I’m pretty sure the second method is faster. It can do simple character compares instead of the more complex string compares and regular expressions have to do at least as much work as skip-chars-forward.

Update: test → tests. Fixed line numbers in text.

Posted in Programming | Tagged | 3 Comments

Xah’s Challenge

A couple of days ago Xah Lee posed a programming challenge. The problem was to write a miniparser in your favorite language to detect mismatched bracketing characters in a set of text files. The set of brackets to be checked were: () {} [] “” ‹› «» 【】 〈〉 《》 「」 『』. His solution is here.

I solved this problem in two “different” ways, both in Emacs lisp. This post is my first solution. I’ll post the second solution tomorrow.

This solution is similar to Lee’s but written in a more functional style. The strategy is to hunt down the brackets with search-forward-regexp, push the opening brackets onto a stack, and check that each closing bracket matches the top of the stack. If it does, we pop the stack and keep going. If it doesn’t, we record the error and write it to an error buffer. If there is anything left on the stack when we finish, that’s also an error because there is an unmatched opening bracket.

The main function, check-file-for-mismatches uses a complicated regular expression to locate the brackets and an alist to type the brackets and identify the matching bracket. In a production system, these would be computed before run time to save on execution time. As Paul Graham has said, the nice thing about Lisp is that all of Lisp is available all the time. That means that we can use Lisp as we are writing the code to compute the regexp and alist. In this example, I compute them at run time so that you can see how I did it.

We start with the list of brackets:

(defvar brackets '(("(" . ")")
                   ("{" . "}")
                   ("[" . "]")
                   ("“" . "”")
                   ("‹" . "›")
                   ("«" . "»")
                   ("【" . "】")
                   ("〈" . "〉")
                   ("《" . "》")
                   ("「" . "」")
                   ("『" . "』")))

and then compute the regexp with:

(defun make-br-regexp ()
  "Make a regexp to check for any of the delimiters in brackets"
  (mapconcat (lambda (bp)
               (concat (regexp-quote (car bp)) "\\|" (regexp-quote (cdr bp))))
             brackets "\\|"))

and the alist with:

(defun make-br-alist ()
  "Make an alist of the brackets. Each entry is of the form:
\(bracket open-or-close matching-bracket\)"
  (reduce #'append (mapcar (lambda (bp)
                             (list (list (car bp) 'open (cdr bp))
                                   (list (cdr bp) 'close (car bp))))
                           brackets)))

Each entry in the alist has the form (bracket type matching-bracket) where type is either open or close and indicates whether the bracket is an opening or closing bracket.

As I said, all of this could be done at coding time and the results made into constants. Instead we define a couple of constants on the fly:

(defconst br-regexp (make-br-regexp))
(defconst br-alist (make-br-alist))

All the work is done in check-file-for-mismatches.

 1:  (defun check-file-for-mismatches (fpath)
 2:    "Check a file for mismatched brackets."
 3:    (let ((fb (get-buffer-create "*Temp*"))
 4:          (stack nil)
 5:          (mismatches nil))
 6:      (set-buffer fb)
 7:      (insert-file-contents fpath nil nil nil t)
 8:      (goto-char 1)
 9:      (while (search-forward-regexp br-regexp nil t)
10:        (let* ((bpos (point))
11:               (brk (buffer-substring-no-properties (1- bpos) bpos))
12:               (blist (assoc brk br-alist))
13:               (btype (cadr blist)))
14:          (cond
15:           ((eq btype 'open)  (push (cons brk (1- bpos)) stack))
16:           ((eq btype 'close) (if (or (null stack)
17:                                      (not (string= (caar stack) (caddr blist))))
18:                                  (push (1- bpos) mismatches)
19:                                (pop stack)))
20:           (t (error "FAIL--found %s instead of bracket" brk)))))
21:      (if mismatches
22:          (mapc (lambda (e)
23:                  (print (format "Mismatch in %s at position %d"
24:                                 fpath e)))
25:                (reverse mismatches)))
26:      (if stack
27:          (mapc (lambda (e)
28:                  (print (format "%s has an unclosed %s at position %d"
29:                                 fpath (car e) (cdr e))))
30:                (reverse stack)))
31:      (kill-buffer fb)))
32:  

The actual checking occurs in the while loop on lines 9–20. We find a bracket with the regexp search, record its position and type and then check to see if it’s an opening or closing bracket. For opening brackets, we just push it on the stack. For closing brackets we check for the matching opening bracket on the stack and record an error if it’s not a match.

On lines 21–25, we print any mismatches that we found in the file. On lines 26–30 we print an error message for any open brackets still on the stack. Almost half of check-file-for-mismatches is concerned with error reporting. It’s worth noting that after the first error the parsing tends to get out of sync and you can get cascading errors but the error report tells you what happened so that you can fix the real errors and rerun the program as a final check.

To run check-file-for-mismatches we need a function to feed it the files and establish the error buffer. The sample function, do-files that I show below checks files with the extension txt in the ~/elisp directory. What you actually do depends on how you are selecting your files.

(defun do-files ()
  "Check a list of files for bracket mismatches"
  (interactive)
  (with-output-to-temp-buffer "*Mismatch Report*"
    (mapc #'check-file-for-mismatches (directory-files "~/elisp" t "\.txt$"))))

Many of the ideas for these functions come from Lee’s Emacs Lisp Tutorial. This page, in particular, has several useful templates for handling files and buffers in a way similar to what we’ve done here.

Posted in Programming | Tagged | Leave a comment

Common iPhone Pins

I’ve written several (1, 2, 3) posts about the analysis of passwords divulged by groups like LulzSec. The results were terrifyingly consistent: 123456 and password were almost always the most frequently used passwords. Now Daniel Amitay author of the Big Brother Camera Security App for the iPhone has some interesting results for iPhone pins.

Because his app has a lock screen nearly identical to the iPhone phone lock screen, he collected (anonymized) statistics on what codes were being used for his app in the hopes that it could tell us something about what how users are selecting their iPhone lock codes. He collected 204,508 codes and calculated their frequencies. Among other things, Amitay found that the top ten most common pins accounted for 15% of all passcodes.

Without reading further, try to guess the top four most popular pins. Hint: It’s not hard. The number one pin is, of course, 1234 followed by 0000. The next two are 2580 (going down the middle column) and 1111. The next 6 aren’t much harder to guess. Another common scheme is to use the user’s birth year.

Amity also looks at the most common digit for each of the four positions. The most common first digit is 1 by a large margin. If you squint a bit, it almost looks as if the pins obey Benford’s law. The second, third, and fourth digits are more uniform.

The post is an interesting, if depressing, read. As Amitay points out, for a random iPhone a cracker has a 15% chance of unlocking the phone before the automatic data wipe feature is activated. Considering that smartphones like the iPhone often contain confidential data or access to confidential data this is bad news.

In a sad endnote, Amitay has had his app removed from the App Store for his trouble. Apparently Apple was concerned that he was harvesting the Apple lock codes or something. Amitay is discussing the matter with Apple and we can hope that he will have the whole thing straightened out soon.

Posted in General | Tagged | Leave a comment

Goodbye Borders

Although it’s a little outside of our usual subject matter, I sadly note the passing of Borders. Yesterday Borders announced that it was canceling its upcoming bankruptcy auction and liquidating the company instead. Although some creditors are objecting, it’s almost certain that Borders will close its 399 remaining stores and sell off its assets.

The story of how Borders arrived at this end is well known. They didn’t take Internet sales seriously and failed to appreciate how ebooks would change their industry. I’ve written before about the challenges facing the publishing industry and my hope that they would avoid the same mistakes that the music industry made. So far, they appear to be oblivious to the new reality. Perhaps Borders can serve as a wakeup call. No matter how much you want your business to stay the way it was, circumstances and customers move on. You either move with them or you end up like Borders.

Like most folks, I mainly get my books on-line and tend to prefer ebooks over the dead tree variety. Still, I have fond memories of browsing at Borders and enjoying a comfortable chair and cup of coffee while I skimmed through a book. I’ll miss the opportunity to do so in the future.

Posted in General | Leave a comment

The 25 Most Dangerous Software Errors

Each year the SANS Institute and the MITRE Corporation team up to survey the year’s most dangerous programming errors. This year’s list, the 2011 Common Weakness Enumeration, was published at the end of June. This is a great resource and anyone writing software for the Web should read it carefully. The document is a long and thorough analysis of the errors that led to exploits in the last year.

The report begins with some guidance for different types of users—programmers new to security, programmers experienced in security, project managers, testers, customers, and educators—on how to use the list. Next there is a brief listing of the errors sorted in various ways, and finally a detailed explanation and sample code for each type of error. Finally there is a list of “Monster Mitigations” that establish some general principles for writing secure code.

The tops 5 errors were:

  1. SQL Injection
  2. OS Command Injection
  3. Buffer Overflow
  4. Cross-site Scripting
  5. Missing Authentication for Critical Function

I find OS Command Injection in second place surprising, but sadly all of the others are as expected. It beggars belief that SQL injection is still such a common exploit today. The same could be said of buffer overflow. Of course, as LulzSec has shown us, there is every reason to expect that SQL injection will be a major attack vector for the foreseeable future.

Again, if you are writing code for the Web, you really need to read and study this report.

Posted in Programming | Tagged | Leave a comment

An Interesting Talk On Guile

I just ran across an interesting talk on Guile Scheme by Andy Wingo. The talk is from 2009, shortly before Guile 2.0 came out. Wingo discusses the new Guile architecture and plans for a native compiler. One of the most interesting parts for me was the discussion of his plans for Guile to replace the Emacs Lisp engine. That’s still in the future but there’s already an implementation of Elisp that runs on the Scheme VM. Wingo claims that it will make Emacs much faster than it is now and will allow Lispaphobes to customize Emacs in any of the supported languages—Javascript, for example.

It’s a fairly long talk (about an hour and 15 minutes) but it’s pretty meaty and worth the time if you want to know about the future of Guile. Wingo is a pretty impressive guy so, again, I recommend this talk if you haven’t already seen it. You might also want to check out his blog.

Posted in Programming | Tagged | 1 Comment