Kitchin on Org-mode

John Kitchin is on a roll. Just the other day, I wrote about his org-ref project and the really great work he is doing in introducing his students to reproducible research via Emacs and Org-mode. Now, he has a new video out on the splendors of Org-mode.

There are lots of videos on Org-mode, of course, but Kitchin emphasizes its use to produce technical papers using the reproducible research method. Kitchin, of course, is an export on the subject matter so the video is definitely worth your time. It’s just over 18 minutes so it won’t take up a lot of your time. Very definitely recommended.

As one of the commenters noted, you can lose a day reading the Org-mode entries on his blog. I’ve spent some time there myself and recommend that as well. Kitchin has been doing some outstanding work on Org-mode so it’s worth paying attention to his blog.

Posted in General | Tagged , | Leave a comment

Automating Git Bisect

I’ve written a couple of times about Git bisect. It’s a way of finding the commit that introduced an error. It works by (essentially) doing a binary search on the commit history. Now, Curtis Poe over at Ovid shows us how to partially automate the process.

The Git bisect process involves testing each candidate commit and marking the result as good or bad. Usually this process doesn’t involve a large number of steps (being a log2 n process) but it can be (in Poe’s words) boring. Happily, you can often automate the process and avoid the tedium. You do this by writing a script that tests each commit and then just tell Git to run the script on each candidate commit and tell you when it finds the commit that introduced the error. See Poe’s post for the details.

This doesn’t always work because sometimes it’s hard to write a script that will reliably detect an error and sometimes because the test may fail in different ways. To help with the second case, Poe provides a perl script that will look for some specific output or, alternatively, a pattern. If you do a lot of testing using Git bisect, you should give Poe’s post a read. Even if you only occasionally use Git bisect, using the automation that Poe describes may save you some time and effort. If you’re a Magit user, the bisect function—including the run command that automates the process—is built in.

Posted in Programming | Tagged , , | Leave a comment

Reflections on Trusting Trust

A reference to Ken Thompson’s fantastic paper Reflections on Trusting Trust popped up yesterday on Hacker News. I’ve written about this paper before but it deserves a periodic mention.

If you haven’t read this paper before, I urge you in the strongest terms to do so right now. Suffice to say that it has been described as “a truly moby hack” and “the greatest hack of all time.” Really, you’ll be glad you checked it out.

Posted in Programming | Tagged | Leave a comment

A Medical Decision Tree

Back in March, I wrote about Edward Frenkel and his explanation of the purported backdoor in the Dual_EC_DRBG random number generator. I also mentioned his book Love and Math: The Heart of Hidden Reality. Now Zygmunt Zając over at FastML has a nice post on one of the stories from the book.

It’s about how Frenkel hand-constructed a decision tree to determine the treatment plan for kidney transplant patients. What’s interesting is that Frenkel has no medical training or domain knowledge. Rather, he worked with a physician to capture the physicians ad hoc decisions and turn them into an algorithm.

The story is interesting to folks like us because of the way he solved it. It’s a method that developers might try when attempting to capture a human decision process into a program. At first he asked the physician a series of question trying to understand the various parameters and how they related to a final decision. That approach was singularly unsuccessful so Frankel took another approach.

He randomly selected some patient records and had the physician ask him questions, which he answered by consulting the patient’s file. By analyzing the questions, the order in which they were asked, and what the follow up questions were, he was able to construct a decision tree that accurately captured the physician’s subjective thought process. After two and a half dozen cases, Frankel’s algorithm was almost as good as the physician at making diagnoses. By the end of the process his diagnoses were 95% accurate.

In the end, the process was good enough to patent. What I like is how simple and direct the process was and yet it produced outstanding results. I really recommend this post: it’s short but clearly outlines an approach to problem solving that you may find useful.

Posted in General | Tagged | Leave a comment

Guess Who Those Targeted Individuals Are

The Targets

Here in the U.S., today is independence day. It’s a day to celebrate our forebears’ refusal to submit to what they considered unjust treatment at the hands of their government. One of their major complaints was general warrants: the notion that the government could enter your home and rifle through your belongings at will. They felt so strongly about it that the prohibition of the practice is enshrined in our constitution as the Fourth Amendment.

Now, sadly, general warrants are making a comeback, at least in the digital realm, in the United States and most other first world countries. We all know this story: the NSA, GCHQ, and other intelligence agencies have decided that they have the right—in the name of national security, of course—to snoop on our communications and digital data without warrant or specific cause.

The NSA for its part insists that, yes, they collect almost everyone’s information but except for targeted individuals that information is flushed within 48 hours or, at most, 30 days. If, like me, you’re inclined to a nasty, suspicious mindset, you might wonder who those targeted individuals are. Your Aunt Millie is sure it’s just Osama Bin Laden and a couple of his friends but the more cynical of us wonder if “targeted individuals” might be a bit more general.

Now we have an answer. One way to be targeted is to read Boing Boing. Well, Boing Boing is vaguely left-wing, I suppose, although still well within the mainstream so why would that get you targeted? Surely, reading an apolitical, technical site won’t get you targeted. It turns out, though, that reading Linux Journal—reportedly considered an “extremist forum” by the NSA—can also get you on the list. What do these sites have in common that excites the NSA’s suspicions?

The answer is, at the same time, shocking and obvious: reading an article on the technical details of TOR, Tails, or other privacy enhancing software is enough to provoke the NSA’s interest. Cory Doctorow reports that one expert suggested that the NSA is trying to separate the sheep from the goats; to split the population into those who know how to protect their privacy and those that don’t. Naturally, those in the first group are suspicious and therefore targeted.

The Sources

If you read the links above you will see that the information is explosive. Almost too good to check as cynical journalist like to quip. So where did this story come from? The story apparently originated on the German site Tagesschau.de (in German, Google translation here). Happily, for those who don’t read German, there is an English language article that extends the Tagesschau.de article on DasErste.de. If you read nothing else, you should read this article, as it explains in detail where the information comes from and includes a link to XKeystore deep packet inspection rules.

There has been speculation that these latest revelations point to the existence of a second NSA leaker as none of this information was included in the known Snowden documents. Bruce Schneier, who has access to the Snowden documents, does not believe this information came from Snowden and believes there is a second leaker.

There’s a lot of information in the above links but you really should read it all. The original Tagesschau article was about the targeting of German sites so this story concerns you whether or not you are an American. If, after you read it, you aren’t infuriated, let me nominate you for The Alfred E. Neuman “What, me worry?” award.

Posted in General | Tagged | Leave a comment

(Lots of) Stuff You Didn’t Know About Emacs and Unicode

Christopher Wellons has another great post on the minutia of Emacs. This time it’s about Emacs unicode pitfalls. Most of us know that Emacs uses UTF-8 as its internal data representation but little more. That’s mostly Okay because almost all the time Emacs does the right thing without us having to worry about it.

It turns out, though, that there are some edge cases that you probably don’t know about. For example, some (non-ASCII) characters display identically but have different code points. That means, for instance, that equality operations can fail. Wellons discusses these and shows how to deal with them.

You will probably almost never have to deal with these cases but Wellons tells you what to look out for and how to deal with them when they do occur. This is a post you should read and bookmark for those times when you need the information it contains.

Posted in Programming | Tagged | Leave a comment

Emacs Introduction and Demo

Howard Abrams has a nice video that serves as an introduction and demonstration of Emacs. Don’t be fooled by the “introduction” part. Abrams doesn’t spend any time on the usual 【Ctrl+p】 to move up a line, 【Ctrl+n】 to move a line down stuff that take up the majority of introductions. Rather, he emphasizes the demonstration aspects and shows the new user some of the advanced possibilities that Emacs offers. Abrams covers features such as

  • Multiple Cursors
  • Expand Region
  • Org Mode
  • Writing simple Elisp functions
  • linum-relative

I hadn’t seen the linum-relative before but it looks as if it could be useful in a lot of cases. The idea is that the buffer is numbered with 0 at the line the point is on and negative numbers for the lines above, and positive numbers for the lines below. That makes it easy to jump a required number of lines. For example, if you want to jump to the line labeled 6 you could type 【Ctrl+6 Ctrl+n】. Most of the time, I use ace-jump-mode for jumps of that sort but I can see how linum-relative could be useful.

Abrams offers some other tips that you might not have seen so the video is definitely worth watching. It’s about 36 and a half minutes so plan accordingly.

Posted in General | Tagged | 1 Comment

SBCL 1.2.1

The monthly release of Steel Bank Common Lisp is out. As usual, it compiled and loaded without mishap. Unfortunately, Slime wouldn’t compile due to a missing slot in the SBCL VM module. I can call it from the command line but not with Slime from within Emacs.

If anyone gets Slime to compile and work with SBCL 1.2.1, please leave a comment and let me know if you did anything special. When I dropped back to 1.2.0, everything worked fine so I don’t think it’s a system problem. This is under OS X 10.9.3.

Posted in Programming | Tagged , , | 1 Comment

Org-ref!

A month ago, I wrote about John Kitchin’s org-ref package. At the time, it was still an alpha-level project but showed tremendous promise. Happily, Kitchin has continued work on the project and it is now in a ready-for-end-users state. It’s still not an ELPA package but it is available at Kitchin’s GitHub. The GitHub entry is interesting. True to Kitchin’s reproducible research sensibilities, the whole package is an Org file written in a Literate programming style that both documents the package and provides the code. If you’re interested in seeing how to build such a file, click on the Raw button to see the Org source.

The latest news is that Kitchin has produced a video that demonstrates the package. You can see the package in action and get a sense for how you’d use it when writing a technical document. A bibliography is one of those things that’s not really all that hard to deal with but they can be fussy and use up a lot of mental cycles on trivial details. Org-ref helps automate all the fiddly bits .

The video is just short of 11 and half minutes so you won’t need to block out a lot of time. Don’t worry about copying down any of the details; those are all in the Org file at GitHub. In a comment to my previous post, Kitchin indicated that they hope to have org-ref integrated into Org itself. I hope that happens.

Posted in General | Tagged , | Leave a comment

How Good We Have It Now

Via Dennis Ritchie’s Website, here’s a 1984 ATT&T pricelist for SysVR2 Unix. If you were an educational institution, you could get a source license for $800 ($43,000 for others). DMR discusses these prices and some of the license restrictions here.

So what does this have to do with us? For $800—even 800 current dollars—we can buy a pretty good computer, load it with Linux (or one of the BSDs) and have a much better system for just the educational price of a Unix license. If you’re young, it’s always been that way but some of us remember when it was an impossible dream.

In 1984 it was still pretty much MSDOS and $3,000 PCs for individual users. The Internet was not widely available and Windows was just starting to be available and probably wasn’t that useful. A few years later everything had changed and anyone who wanted to could have their very own, full-featured Unix-like system. These days, even non-geeks can enjoy all the benefit of Unix nicely hidden behind the Mac OS X GUI.

We really do have it pretty nice now.

Posted in General | Tagged | Leave a comment