Flash and Security

In an editor's note to the SANS Newsbites piece on Flash being used as a vector for the ransomware CryptoWall, William Hugh Murray speaks the uncomfortable truth on what Flash tells us about security.

Posted in General | Tagged | Leave a comment

Emacs and Data Science

Robert Vesco has an interesting post on why he uses Emacs in his data science work. Vesco is a data scientist for Bloomberg so he's a serious practitioner of the art. Working in data science means he uses a variety of languages such as Python, R, SQL, Stata, and SAS. He notes that most of those languages have an associated IDE that simplifies working with them but that that means learning multiple editors and probably mastering none. He also notes that those specialized IDEs may fall out of favor and not be supported in the future, that they are not portable across platforms, and that they are hard to customize.

Happily, those defects do not apply to Emacs. It runs on essentially every (serious) platform, is open source, will be supported for as long as there are a few programmers still interested in using it, and, of course, is famously customizable. One consequence of that customizability is it can become a reasonable IDE for almost any language. That means that a single tool can be used for all those languages and that it's worthwhile mastering that tool because it's the only (editing) tool you need to learn.

The bulk of Vesco's post covers those features of Emacs that he finds most useful in his work. One of those features is Org mode that allows him to use reproducible research methods in his research and publishing. It's an interesting read even if you're not a data scientist.

Posted in General | Tagged , , | Leave a comment


I learned a really useful Org mode command from Artur Malabarba's first anniversary post: org-goto. The idea is that you want to navigate to somewhere else in the current Org buffer. You could fold the buffer and navigate to the proper subtree but often you want to leave the current subtree unfolded as well as the subtree you're navigating to. The org-goto command, bound to 【Ctrl+c Ctrl+j】 by default, allows you to do just that. An copy of the buffer is created with an overlay displaying the buffer in overview mode. You can navigate within that buffer and press 【Return】 on the desired subtree which will take you back to the original buffer with the point at the new subtree.

That may not seem too useful but see Malabarba's post for a compelling use case. In my case, it's useful because of the large Org files I use to store my data. When I load such a file via a bookmark or it gets reloaded during a sync from git, it will be completely folded. The files are too large to set them to be completely unfolded so I have to have some way of finding the proper place. I used to call org-sparse-tree and then do a regular expression search for the proper heading. With org-goto, I can simply display the headers, navigate to the desired one, and press 【Return】 to navigate to the proper heading that will be conveniently unfolded. Very handy.

It's a bit hard to explain how org-goto works so you should experiment with it a bit to see how you can fold it into your workflow. If you use Org files to organize your data as I do, this command is very likely to be a win for you.

UPDATE: There's a really good discussion of org-goto and related navigation methods in the comments. Be sure to take a look.

Posted in General | Tagged , | 13 Comments

Making My Org Agenda More Useful

I've written before (1, 2, 3) about my efforts to implement a system like the one Karl Voit has for managing personal data. Voit uses his Memacs system to automatically collect data from a variety of sources and add them to Org files so that they will appear in his Org agenda. Unfortunately, Memacs was written with the Linux/Android ecosystem in mind so much of it needs a bit work to be usable in the OS X/iOS world.

Nevertheless, many of the ideas (and probably some of the scripts) from Memacs can be used with Org mode on any platform. I keep most of my personal data in 5 Org files: journal, chart (a lightweight quantified self), b-ideas (a queue of ideas and notes for blog posts), todo, and notes. I have capture templates for each of these making it easy to add data. My latest effort to make this data more accessible and useful was to modify the capture templates to add an active time stamp to each entry, causing it to appear in that day's agenda. For example, here is the capture template for my journal:

("j" "Journal" entry (file+datetree "~/org/journal.org")
         "* %<%R: >%? %^g\n%t")

The %t at the end causes an active date stamp to be added to each journal entry. Thereafter, the headline and tags from the entry will appear in the agenda and selecting that headline will take me directly to the entry.

Here's an example of one day's agenda. If I want to know what happened on that day, I can jump directly to it in my agenda to find out. If I want to see where I've had dinner, I can bring up all the entries with a tag of dinner. It's a tremendously powerful system that comes almost for free just by arranging for those active date stamps.


It could be better though. That's where Memacs comes in. It automatically extracts data from SMS, the Phone's log, posts, emails, and other data sources. All of the data magically appears in the agenda without having to explicitly capture it. My next step is to adapt some of Voit's Memacs scripts for my own use.

All this seems pretty geeky but I use the system every day for practical tasks. This morning, for example, I used it to extract data for my tax files. If, as often happens, I find I need to make an update to a blog post, there's a link to the source file right in the agenda and clicking on it brings up the proper file. As you can see in the example, I also add technical items to my journal. If I need to export an Org file containing citations to docx and don't remember how, I can search for the docx tag to find the journal entry that contains the details.

Org is tremendously powerful and it pays huge dividends for the effort you expend learning it. One of the nice things about Org is you don't have to learn everything at once. I keep learning new bits as I find a need for them.

Posted in General | Tagged , | 2 Comments

What is a Hacker?

That's a notoriously difficult question. Nik Ferrier has his own answer, which seems to me as good as any other I've read. Hop on over and see if it resonates for you.

Posted in General | Leave a comment

Org-Babel Examples

I haven't written about using Babel with Org mode for a long time but it's something I use everyday. Most of those uses are trivial but the full power is there when I need it. I can produce summary data and graph that summary almost effortlessly.

If you're looking for some examples of what you can do with Org and Babel, Derek Feichtinger has a Github repository with examples using several languages. It's well worth looking at because Org with Babel is the key to writing your documents in a reproducible research way.

You may not be writing journal articles that other researchers will want to check but even if you're just documenting some aspect of your own work, having a document with all the code embedded in it is tremendously powerful. Right now, I have legacy documents where the code that process their data are separate applications. I also have Org mode documents where the code to process the data is embedded in the document and I can update the results simply by typing【Ctrl+c Ctrl+c】on the code block. I can tell you that the latter documents are far easier to deal with.

Check out Feichtinger's collection of examples to see if some of them won't be useful to you.

UPDATE: Added link to Feichtinger's Github repository.

Posted in General | Tagged , , | 2 Comments

Schneier on Why We Encrypt

Bruce Schneier has a nice post on Why We Encrypt. Encryption maintains our privacy, keeps our money safe, and, sometimes, saves our lives. Most Irreal readers will be familiar with these reasons but here's another that we hear about less often: if you only encrypt important data, you are putting a sign on that data saying, “this is something worth trying to exploit.”

The answer, of course, is to encrypt everything. The movement to deprecate HTTP and use only SSL/TLS (HTTPS) is part of that. If all of our Web transactions are encrypted, it's hard to know which ones are worth trying to decrypt. The other, harder, major vector is email. I hardly ever encrypt email because almost no one I communicate with is prepared to deal with it.

That's a long standing problem, of course, and one that doesn't admit an easy solution. Schneier notes that encryption works best when it's automatic. That's what we need for email. A system that automatically encrypts the messages we send and decrypts them (also automatically) at the other end. That way, Aunt Millie doesn't even have to know what encryption is; everything is handled behind the scenes. We're still some way from that but I'm looking forward to the day when all the three-letter agencies are sad because it's really hard to snoop on people anymore.

Posted in General | Tagged | Leave a comment

Happens to Me Every Day

Happily, with OS X you can make a lot of the Emacs key chords work in other apps but it's still not the same.

Posted in General | Tagged | Leave a comment

Annual Who Has Your Back Report

The EFF has published its annual Who Has Your Back? report. The idea is that they look at those technology companies that deal with our private data and rate them on the following criteria

  1. Follows industry-accepted best practices
  2. Tells users about government data demands
  3. Discloses policies on data retention
  4. Discloses government content removal requests
  5. Pro-user public policy: opposes backdoors

For better or worse all but the most paranoid of us use several of these companies (7 in my case) so it's worth knowing which ones are looking out for their users and which are selling our data or rolling over for the government.

If you're looking for the TL;DR, the chart at the beginning of the report will tell you who the good guys are and how the others are failing. If you're looking for a deeper dive, each company's performance is discussed in detail later in the report. There are also links to the policies and transparency reports for each of the companies.

Most people probably aren't going to stop using a company that doesn't rate a perfect score but it's useful to know who's doing what and to adjust your behavior accordingly. It's an interesting report and everyone should at least look at the chart. Unless, that is, you're one of the paranoid that stays off the grid. Of course, in that case you probably aren't reading this.

Posted in General | Tagged | Leave a comment

A Tip for Artist Mode

If you use artist mode in Org mode, you may find this tip useful.

Posted in General | Tagged | Leave a comment