Keyboard Macro Counters

Emacs is incredibly useful for repetitive editing chores. For example, this Xah Lee post shows how Lee performed major surgery on hundreds of pages of HTML by using a little Emacs Lisp.

Not everyone can or wants to write Emacs Lisp but even without it you can still do a lot of automation with keyboard macros. Most Emacs users are familiar with the fundamentals: use 【Ctrl+x (】 or 【F3】 to start recording your keystrokes, perform the repetitive action one time and stop the recording with 【Ctrl+x )】 or 【F4】. Then you can repeat the keystrokes with 【Ctrl+x e】. After the first repeat you can simply type 【e】 for subsequent repeats.

Here’s a little trick I learned from the EmacsWiki. Suppose you want to create a list of host names where each differs from the others by a number at the end

host1.irreal.org
host2.irreal.org
...

To do that, follow these steps

  1. Ctrl+x Ctrl+k Ctrl+a1 to set the amount to increment the counter on each repetition.
  2. Ctrl+x Ctrl+k Ctrl+c1 to set the initial value of the counter.
  3. Ctrl+x (】 to start recording the macro
  4. hostCtrl+x Ctrl+k Ctrl+i】 to insert the current counter value after host.
  5. .irreal.orgReturn Ctrl+x )】 to stop recording the macro.
  6. Ctrl+x e】 to generate the next host name.
  7. e】 to generate another and so on.

You can also use 【Ctrl+x Ctrl+k Ctrl+f】 to set the format of the counter.

Posted in General | 1 Comment

Git By Hand

I went looking for the update to Tom Preston-Werner’s talk on Git that I wrote about previously but I didn’t find it. What I did find, however, is another talk on Git, entitled Unlocking the Secrets of Git that he gave at the UCAR Software Engineering Assembly. This was a very enlightening talk that along with his Git Parable provides an excellent understanding of the Git architecture and how it works under the covers. He describes Git as a user-land file system with versioning and synchronization.

In this talk he constructs two Git file blobs by hand (using an editor and an interactive Ruby shell) and shows how they get named by their Sha1 hashes. Then he constructs a Git tree object—essentially a directory listing of the two file blobs—and then uses git cat-file to show its structure. Finally, he makes a commit object and again shows what it looks like with git cat-file.

This talk really brings alive how simple Git is. There’s something about seeing the objects made by hand (not even using Git in the case of the file blobs) that dispels all the mystery. I really can’t recommend this talk enough if you’re a Git user. It will really help you understand what’s going on.

Of course, there’s a lot more to Git then the add, and commit commands so the talk won’t make you a Git master but it will give you the background to understand the more advanced commands and how they fit in with the Git way of doing things.

Preston-Werner is a Git master and a cofounder of GitHub so he’s a good teacher and give gives an entertaining talk. It’s about 75 minutes long so you’ll need to block out some time but it’s well worth an hour and a quarter of your attention.

Posted in General | Tagged | Leave a comment

SSH Tricks

Smylers over at the Smylers Blog has an outstanding post on little known things you can do with SSH. Even after years of using SSH daily, there were still a couple of tricks new to me in the post. Smylers mostly assumes you’re using OpenSSH, which unless you’re a Windows user you probably are. Even so, he shows how to do the same things with Putty or Copssh, so this is a worth while read for anyone who connects to a remote system.

Among the things he shows you are:

  • How to share a single remote connection with several terminal windows. That’s handy when you want to have several views into your work on the remote server but don’t want to go through the pain of authenticating several times.
  • How to SCP over an existing connection.
  • How to make an SSH connection persist even if you log out.
  • How to set up an SSH key so that you don’t have to give your password when you sign on.
  • How to move that key to the remote system automatically. This is a real time saver because doing it manually is a pain.
  • How to forward connections from one remote server to another.
  • How to set up host aliases so you don’t have to type a long fully qualified host name.
  • How to configure the user name you use on the remote host if it’s different from your user name on your local host. This removes the necessity to specify your login name when you connect.
  • How to forward X11 connections.
  • How to mount remote files locally over an SSH connection.
  • How to connect to remote services with a local application. For example, if you have a database on a remote system that expects a connection on a certain port on the remote system you can configure SSH to do port forwarding to enable this.

There are some other tricks as well. Some of the commenters offer additional tricks so be sure to read them too. This is a really great post and if you’re an SSH user you should read it without delay. It has some techniques that will really make your work flow easier.

Posted in General | Tagged | Leave a comment

How Passwords Get Stolen

I just stumbled across a nice post on How Attackers Steal Passwords by Joe Golton over at FilterJoe. It’s an interesting look at the common attacks on user passwords. There’s not a lot new or surprising in the post but it’s a good summary of the various attacks and what users can do to avoid or mitigate them.

The attacks fall roughly into 3 groups

  1. Social Engineering
  2. Attacks on the Web site servers
  3. Malware that the user downloads that then sets up a keystroke logger or other attack.

There’s not much we can do to protect ourselves against the second type of attack except to never reuse passwords and to avoid dealing with companies and sites that have demonstrably poor security practices. Avoiding social engineering attacks is mostly a matter of common sense and staying alert to the danger. Golton recommends just deciding in advance that you will never give anyone your passwords under any circumstances. If you are forced to do so in order to perform some needed chore—to import data, say—then immediately change it afterwards.

The third category is the hardest to deal with. Some password managers can help here with their automatic log on functions, which makes it difficult for keystroke loggers to intercept the passwords. Browsers like Chrome that set up sand boxes can also help but as Golton explains, some of these attacks are Javascript based and so operate entirely within the browser. The safest thing to do is to use a separate machine for banking and other high value transactions but not many people are going to do that. At the least, it may pay to quit your browser and start a new instance before doing any banking in order to get rid of stray Javascript.

Golton’s final note is that six of the nine attacks can be blocked with the correct use of a good password manager. He recommends RoboForm (Windows), 1Password (Mac OS X), KeePass (free), or LastPass for those looking for a cloud-based solution. You can read his reasons for these recommendations here.

Posted in General | Tagged | Leave a comment

Administrivia: Captchas

The amount of comment spam has reached intolerable levels: spam comments far outnumber legitimate comments and moderating them is taking an increasing amount of time. Therefore, I have reluctantly installed a captcha to try to bring the problem under control. I’m using the NUCaptcha plugin, which features moving letters that are easier to read so I’m hoping that this won’t inconvenience folks too much.

The captchas appear only to unregistered users so if you are a regular commenter, it’s probably worth your while to register and avoid the hassle of dealing with captchas. Again, sorry for the inconvenience. As usual, a few bad actors ruin things for everyone else.

Update: inconvience → inconvenience

Posted in Blogging | Leave a comment

Emacs Ruler Mode

Sometimes it’s convenient to have a ruler on your Emacs screen. Not often but sometimes. For example, it can be useful for lining things up in artist-mode. Emacs has a very nice ruler that, among other things, tracks were the point is on the line.

You can toggle it on and off with 【Meta+xruler-mode. Often times you (or at least I) want to turn it on for just a second to check something so you can turn it on and then type 【Meta+x Meta+p】 to bring the command up again to turn it off.

This isn’t a command you will use a lot but when you need it, it comes in very handy. If you forget its name, 【Ctrl+h fruler will bring up the on-line documentation.

Posted in General | Tagged | 2 Comments

Emacs Vs

I’ve just run across an interesting site created by Strzelewicz Alexandre called Emacs Vs or Emacs Vote System. Users post their favorite Emacs tricks and the other users vote them up or down—much like Hacker News or reddit. The current favorite is yasnippet, a template system for Emacs.

This site is worth adding to your RSS feed just to keep up with the new submissions. If, like me, you’re always on the look out for new Emacs tricks, this could be a good place to find them.

Posted in General | Tagged | Leave a comment

Diceware Implementation (Part 2)

Last time I presented an implementation of the Diceware method for generating a secure password. Today I’d like to finish up with a few details.

First, the careful reader might wonder why I generated 4 random bytes with RAND_bytes instead of 2. After all, 2 bytes is more than enough to express 7,776 so we’ve wasted two bytes worth of entropy. The reason is that the modulo operation gives a slight bias against the higher numbers and this bias is more pronounced when the modulus and largest random integer are closer together. For example using 2 bytes, the 65,536 integers results in the numbers 0–3,326 being generated 9 times but the numbers 3,327–7,775 are generated only 8 times. That means that the lower numbers are generated 1.125 times as often as the high numbers. That’s a significant bias.

By using 32-bit random integers we get the numbers 0–2,558 generated 552,337 times and the numbers 2,559–7,775 generated 552,336 times. Thus, while there is still a bias it is much less: the lower numbers are generated 1.0000018 times more often than the high numbers.

Second, Apple has deprecated RAND_bytes as of Lion but Google doesn’t know why or what they anticipate will replace it (or at least my Google-fu is insufficient to discover these things). If anyone knows what’s going on here, please leave a comment.

Third, RAND_bytes uses /dev/urandom to seed the PRNG. If your system doesn’t have /dev/urandom (RAND_bytes will return 0) you must seed it yourself with RAND_add.

Fourth, compile the C code with

gcc -Wall -lcrypto diceware.c

Finally, a word about master passwords. As I said in the previous post, you really want to have one master password that is strong but reasonably easy to remember that protects all your other passwords. The question is, how do you store those other passwords safely? The obvious answer to is use a password manager such as 1password or KeePass but if you don’t want to spend money or deal with third party software, you can keep them in an encrypted Org file as explained in this Minor Emacs Wizardry post. This is what I do and it works out fine except that it isn’t integrated into my Web browser the way some of the password managers are. If you roll your own with Emacs and Org mode, make sure you generate strong passwords for your other accounts. You can use something like the makepw utility that I wrote about in my old blog, for instance. If you do use makepw, you might want to have it generate 20 characters instead of 10.

In any event, once you choose a way to manage your passwords, you can use a master password generated by the Diceware method.

Posted in Programming | Tagged | 1 Comment

An Implementation of Diceware

A few of my recent posts (1, 2, 3) discussed the Diceware method of choosing a password. The idea is that you roll a die 5 times to get a 5 digit number and use that number to look up a word in a list of 7,776 words. If you do this 5 or 6 times you end up with enough random words for a strong password that is reasonably easy to remember. All of this discussion, here and elsewhere on the Web, were precipitated by this XKCD cartoon, which explains the rationale for the idea well.

As an amusing little side project that might be useful to some, I implemented a software version of the Diceware method. You just run the utility specifying how many words you want and out pop that many words. Although the Diceware page recommends against doing a software implementation, mine uses a cryptographically strong pseudo-random number generator and should be at least as secure as actually rolling a die.

Before we begin, there are 2 points worth mentioning:

  1. This is a good way to generate a “master password” that is used to protect your other passwords (I’ll say more about this in the next post). As Troy Hunt pointed out you shouldn’t use this method or any other to try to remember all your passwords.
  2. People find it very difficult to believe that 5 random words from a list of only 7,776 could constitute a strong password. One commenter on Hunt’s post even claimed that such a system could be broken in a matter of seconds. The key here is that the words are chosen at random. How many ways are there to choose 5 words from that list? Well, you can choose the first one 7,776 ways, and the second one 7,776 ways and so on so that there are 77765 = 28,430,288,029,929,701,376 ≈ 264 ways. That is, those 5 words have 64 bits of entropy. Because they were chosen randomly there are no patterns to exploit so the entire search space must be examined. How long would that take? Assuming (very generously) that you could check 1,000,000 passwords a second, it would take over 900,901 years. Throw in another word and it would take over 7,005,409,781 years.

First, I downloaded the word list from the Diceware site. A typical line of the list looks like

21124 clip

so I cut out the numbers leaving just the list of words, one to a line, and renamed it diceware.txt.

Here’s the actual C code.

 1:  #include <stdio.h>
 2:  #include <string.h>
 3:  #include <stdint.h>
 4:  #include <openssl/rand.h>
 5:  
 6:  #define NWDS    7776
 7:  #define MAXWDSZ 10
 8:  #define ERROR(m) {puts(m); exit(1);}
 9:  
10:  /******************************************************************************
11:   * get_words -- read the dice words into an array                             *
12:   ******************************************************************************/
13:  
14:  void get_words( char **wds )
15:  {
16:      static char buf[ NWDS * MAXWDSZ ];
17:      char *s;
18:      char *p = buf;
19:      int i;
20:      int len;
21:      FILE *dicewds = fopen( "diceware.txt", "r" );
22:  
23:      if ( !dicewds )
24:          ERROR( "Couldn't open diceward.txt" );
25:  
26:      for ( i = 0; i < NWDS; i++ )
27:      {
28:          s = fgets( p, MAXWDSZ, dicewds );
29:          wds[ i ] = s;
30:          len = strlen( s );
31:          *( s + len - 1 ) = '\0';
32:          p += len + 1;
33:      }
34:  }
35:  
36:  /******************************************************************************
37:   * main -- generate n random Diceware words                                   *
38:   ******************************************************************************/
39:  
40:  int main( int argc, char **argv )
41:  {
42:      char *wds[ NWDS ]; /* list of Diceware words */
43:      int i;
44:      int dwords;        /* number of words to generate */
45:      int ix;            /* random index into list of words */
46:      union
47:      {
48:          unsigned char rand_buf[ 4 ];
49:          uint32_t rand_int;
50:      } u;
51:  
52:      if ( argc != 2 )
53:          ERROR( "Incorrect argument count" );
54:      dwords = atoi( argv[ 1 ] );
55:      if ( dwords <= 0 )
56:          ERROR( "Incorrects number of words requested" );
57:      get_words( wds );
58:      for ( i= 0; i < dwords; i++ )
59:      {
60:          if ( !RAND_bytes( u.rand_buf, 4 ) )
61:              ERROR( "Couldn't get enough entropy for secure generation" );
62:          ix = u.rand_int % NWDS;
63:          puts( wds[ ix ] );
64:      }
65:      return 0;
66:  }

The get_words function on lines 14–34 reads the list of words into the wds array and chops off the newlines. Surprisingly, get_words is more complicated than the actual random word generation that takes place in main (lines 40–66).

First the number of words to generate is retrieved from the command line and put in dwords on line 54. Then get_words is called on line 57 to fill the wds array. The actual generation take place in the for loop on lines 58–64. First RAND_bytes is called to get 4 cryptographically strong random bytes. Those four bytes are reduced modulo 7776 to get a number (in ix) between 0 and 7,775. Finally, the ixth entry of the wds array is printed. This is done dwords times.

This post is already too long, so next time I’ll finish up with a few technical details.

Update: chops of → chops off

Posted in Programming | Tagged | Leave a comment

Most Of Let Over Lambda Is Now On-line (Mostly)

I’ve been wanting to read Doug Hoyte’s Let Over Lambda for some time but I’ve held off because I’m trying to avoid buying physical books. I kept thinking that it would appear as an ebook on Amazon or iBooks but it never did. Now, however, Hoyte has posted most of the book on his Web site. The first 6 chapters and references are available. The last two chapters and appendices have not been released.

While the whole book is not (yet?) available on-line, there’s plenty to get me—and you—started. I plan to get started right away. If Hoyte decides never to release the rest of the book, I can always break down and buy a physical copy but I’d much rather have a PDF or even one of the ebook formats.

I’m not sure why he hasn’t released an ebook. I notice from a Web search that there is already a (presumably bootleg) PDF available so it’s not as if he would be losing revenue. He would, I think, gain it as people like me who are trying to avoid reading off dead trees but still want to support the author flocked to buy it.

In any event, I’m delighted that much of the book is available and am looking forward to reading those 6 chapters.

Posted in Programming | Tagged | Leave a comment