Speeding Up Org File Loading

Mahmood over at λ mental has a surprising post concerning the time to load and parse Org files. What was surprising to me is that a method I would have expected to take longer was actually much faster. Here’s a couple of statistics from Mahmood’s post:

Command Time
(find-file-noselect "/home/mahmooz/brain/notes/1707069432.org") 0.212753316
(quick-way) 0.003604351

where quick-way is

(defun quick-way ()
  (with-temp-buffer
    ;; the final `t' is to replace contents of the temp buffer
    (insert-file-contents "/home/mahmooz/brain/notes/1707069432.org"
                          nil
                          nil
                          nil
                          t)
    (let ((major-mode 'org-mode))
      (org-element-parse-buffer))
    nil))

As far as I can see, after a quick scan, find-file-noselect calls insert-file-contents after a bit of preprocessing but nowhere near enough reprocessing to account for the huge time difference. My unconsidered expectation was that it would be the other way around (insert-file-contents calls find-file-noselct), which explains my confusion. But I still don’t understand where the difference is coming from. Doubtless if I bore down a bit or (gasp) actually did some profiling I could find out but I’m enjoying being mystified and—truth to tell—am too lazy to track it down.

In any event, that little mystery is only part of Mahmood’s post. His main complaint was that loading his Org agenda and Org Roam files was taking too long. The above observation turned out to be the crux of the matter and Mahmood provides some code to load and parse Org files much faster than the default methods.

Posted in General | Tagged , | Leave a comment

Mental Challenges And Emacs

JTR over at The Art Of Not Asking Why has a post that explores why Emacs is the right thing. He begins with a story of a Porche owner who often has to drive in countries that require cars to have right hand drives1. Rather than give up their beloved Porche, they simply modified the car to have an adjustable steering wheel that could be set to either side.

JTR, of course, compares this to using Emacs. Others may lay down restrictions but with Emacs you can simply change it to adapt to those restrictions. As JTR says, sadly, for a lot of apps, configurabilty boils down to choosing between light and dark modes. Those apps force you to work the way they want you to. They give no thought to how you like to do things; they provide a solution and you have to use it whether you like it or not. These apps present a mental challenge: you have to do things that aren’t natural to you in order to work with them.

An example that JTR gives is date formats. Apple apps and many others prefer the US date format but the YYYY-MM-DD format is clearly superior. For one thing, it will sort dates correctly. I think of it as being like the Chinese personal naming convention of Last Name followed by first name. It’s the same principle of stating things in order of more general to more specific.

The main point is that Emacs allows you to adjust it to work the way you prefer. That’s sadly lacking in the majority of applications.

Footnotes:

1

JTR speculates that this is the UK or Ireland but although they both drive on left side of the road there is nothing that requires a car to be a right hand drive. Perhaps there are other regions that do require this but I’ve never heard of one. More likely, the story is a metaphor.

Posted in General | Tagged | Leave a comment

Casual Ediff

Ediff is a thing of beauty. It provides a way of not only seeing the difference between two files but for merging various differences into a third combined file. It’s very powerful and flexible. The problem is that using it can seem extraordinary complex. There’s a command menu that may or may not appear in your current frame but even if it does there are a plethora of options most of which are unfathomably opaque.

Charles Choi is once again easing our pain by providing a transient menu that tries to bring sanity to the confusion. His new package, casual ediff, tries to cut through the confusion. Take a look at his post to see causal ediff in action.

Ediff is really powerful and every Emacs user should be able to use it. Choi’s casual ediff can help with that but as I learned from this video from Prot, you can ignore most of the complexity of ediff. There are really only a very commands that you need to know. As I said in my post about Prot’s video:

All you really need to know is n for the next diff, p for the previous diff, and a, b, or c to move the A, B, or C diff to the other buffer(s). That’s it. It covers almost everything you ever really want to do in Ediff.

If you’re casual suite user, by all means include casual ediff in your repertoire but if you’re not and just want to make using ediff as simple as possible, take a look at Prot’s video.

Posted in General | Tagged | Leave a comment

Sharpening The Saw

Christian Tietze has a nice post on sharpening the Emacs saw as part of this month’s Emacs Carnival on Maintenance. Like many of us, Tietze found that he was moving more and more of his work into Emacs. Finally, he found himself with a bespoke Emacs that was his alone.

Once you have your very own personalized Emacs, you have to maintain it. It’s not necessarily that way for other editors. I was a Vim user for over two decades and very rarely did any maintenance other than compiling and installing new versions. With Emacs, it seems as if I’m always tweaking some setting or adding a new package.

Tietze compares this to taking care of your tools and work area in, say, a woodworking shop. As he puts it, you can’t cut with a dull saw. You need to take care of your tools, not just use them. Equally important, you need to think about how you use your tools and what you’ve accomplished with them in the past.

Tietze says that he considers the time spent tinkering with Emacs and personalizing his computing environment as an investment in his future. It’s like sweeping out and organizing the shop and keeping everything neat and tidy so that you can work unfettered by distracting clutter.

Tietze makes a point I’ve often made myself. Once you shape Emacs into the computing environment that’s specialized for you, you have a perfect tool that you can take with you wherever you go. In our environment, that means whatever operating system or computer you’re using. Emacs runs just about everywhere.

Posted in General | Tagged | Leave a comment

Delete Pair

Courtesy of a post from the_cecep over at the Emacs subreddit, I just discovered a builtin gem I didn’t know about. The post is about the delete-pair function. The idea is that you have some text surrounded by a pair of delimiters and you’d like to delete the surrounding delimiters.

That’s something that I sometimes want to do but it gets better. The_cecep shows how to quickly change (foo) to [foo] with delete-pair. That’s something that I want to do less frequently than simply deleting the delimiters but it’s still useful.

Take a look at the_cecep’s post for some more examples and a few details. He also recommends that because delete-pair is unbound you should bind it to Ctrl+Meta+z, which he thinks of as “zap pair”. That seems like a good suggestion to me.

There’s a couple of things worth noting:

  • Although the post doesn’t make it clear, you don’t need to select the expression if you just want to delete the delimiters.
  • You can set delete-pair-blink-delay to 0 to disable the delay if you don’t want it. The default is 1 second so you should at least set it to a smaller value as the_cecep suggests.

I’m really happy to learn about delete-pair. It’s not something I’m going to use every day but I will use it often enough to internalize it.

Posted in General | Tagged | Leave a comment

Piping In And Out Of Emacs Buffers

A couple of weeks ago, I wrote about piping Eshell output to a buffer. It’s a cute trick but it only works in Eshell. It turns out you can do better.

iLemming has a video on piping in and out of an Emacs buffer even in a normal shell. He begins by showing how to accomplish input redirection within Eshell. That’s not supported by Eshell itself so it requires a supporting eshell function—that iLemming calls b​—to work.

The real juice of the video is making this work in a normal—that is, non-Eshell— shell. That requires another function, mxp, this time a shell script. If you watch the video or follow the mxp link, you’ll see several examples of mxp in use. It’s really quite flexible.

All of this clever and the code is worth taking a look at to see how you might do this type of thing. Still, I don’t see how I would use this capability. I could probably make up some fake application for it but it’s hard to imagine any realistic use. Of course, that’s probably due to my lack of imagination and I’m sure some of you will jump in to educate me.

Update [2025-11-07 Fri 10:49]: Ag Ibragimov notes in the comments that he’s moved the project because of a naming conflict. The new GitHub repository is here.

Posted in General | Tagged | Leave a comment

Thompson’s Reflections on Trusting Trust revisited

As Irreal oldtimers know, My absolute favorite Computer Science paper is Ken Thompson’s Reflections on Trusting Trust. It is truly a thing of beauty. This is a beauty that should be appreciated in stages.

In the first stage, we’re introduced to, as the Jargon File puts it, a truly moby hack perpetrated by Thompson. He changed the C compiler to recognize when the login function was being compiled and insert a backdoor that would allow anyone knowing a special password to login. There’s nothing special about that, of course but the clever part was that he also changed the compiler to recognize when it was compiling itself and to insert the code that generated the back door. He used that compiler to compile a version of the compiler without the changes. Now he had a compiler that would insert the backdoor and replicate the code to insert the back door if the compiler itself was recompiled. But the compiler sources no longer showed any signs of this.

The second stage is understanding how this was done. It turns out to be pretty simple. Thompson worked at the source level by changing the C preprocessor so that he could work at the source level of the code being compiled. The login backdoor insertion code simply looked for some specific code that called crypt and installed the book door there.

The really clever part was installing the code to replicate the backdoor insertion code in a new compiler. Keep in mind that the source for that was no longer present in the C compiler source. Think, for a moment how you would do that. Remember, you’re working at the source level. The answer has to do with quines.

Back in 2023, Russ Cox asked Thompson if he still had a copy of the code for the hack. Thompson did and he sent it to Cox who wrote an excellent exegesis of it. I wrote about his explanation here. Now Micah Kepe has revisited that code and provided his own explanation of it. Take a look at the explanation and try to understand it. You’ll have to bear down but once you understand the details, a whole new world of understanding will be open to you.

The final stage of appreciation is realizing that it’s really hard to be sure your computing environment is secure. Even if you’re paranoid or use a distribution like Gentoo in which you compile everything by default, you still can’t be sure someone hasn’t slipped in a backdoor. There are, of course, ways of mitigating Thompson’s type of attack but who knows what else is out there?

Posted in General | Tagged , | Leave a comment

Oil.el

It’s a lazy Sunday and there’s nothing much going on in the Internet to interest Irreal or its readers so I’m offering you this little nugget that popped up this morning. Oil.el is a tiny package that brings the functionality of Oil.nvim to Emacs.

The idea is that you can batch create files in a given directory by simply listing their names, one per line, into the app. When you’re done, simply type Ctrl+c Ctrl+c and the files are created. Or, you can cancel the operation with Ctrl+c Ctrl+k.

It’s not clear to me why this app is as popular as it seems to be—at least in the Vim world. It’s not that there’s anything wrong with the app itself, I just don’t see the need for it, especially given the existence of Dired. Nonetheless, Emacs is all about letting you have it your way so if Oil’s workflow appeals to you, its existence is a good thing.

The package is not (yet?) on Melpa but you can download it directly from GitHub if you’re interested. I’m guessing that this app will be mostly of value to refuges from Vim who are used to its Vim analog. Regardless, it’s there for you if you’re interested.

Posted in General | Tagged | Leave a comment

Bending Emacs 4: Batch Renaming Of Files

Álvaro Ramírez has a new video up. This one, Bending Emacs 4, demonstrates several ways of batch renaming files. The secret sauce in most of these methods is the ability of the Dired buffer to become writable and have the changes reflected in the underlying file system. Every time I use it, it seems like magic.

Ramírez’s video shows several ways of performing bulk renamings in the Dired environment. Probably the easiest—or the most natural to me—is simply using a keyboard macro. For instance, if you have a directory full of files ending in .org and want them all to end in .txt, you simply start recording a macro, change the first file, move to the next line, and close the macro. Then you can run the macro on the rest of the buffer to rename all the files. After the renaming, all you need do is type C-c C-c to instantiate the changes into the file system.

Ramírez’s favored way of making the changes is to use Magnor Sveen’s wonderful Multiple Cursors package. It’s a bit more flexible than a keyboard macro because you have finer control over which lines will be changed. Once you’ve got the desired files marked, you simple make the change on one and they are all affected.

The video is only 6 minutes, 42 seconds long so it should be easy to fit in. Ramírez even includes a DWIM command line method of renaming the files. The nice thing about using Dired is that you can arrange for Git to be aware of the renaming. Take a look at the video for the details.

Posted in General | Tagged | Leave a comment

Red Meat Friday: AWS Is Down, I Can’t Sleep

Farce comes to the Internet. All right, farce comes to the Internet again. With the failure of AWS this week, many, many services were interrupted, including my Spanish learning app. It was annoying but, after all, you can always practice Spanish or do your banking, or whatever a little bit later.

But one group of users is especially aggrieved. It turns out that some people actually have their mattress connected to the Internet. What could go wrong? Apparently, a bunch of people found out with the AWS outage. So called “smart mattresses” monitor your sleep patterns and can adjust the mattress temperature on the fly.

Imagine the consternation from the purchasers of these $2000 mattresses when their sleeping temperature is disrupted. How can one possibly get a decent night’s sleep if the mattress isn’t adjusting the temperature according to our sleep patterns? Of course, the vast majority of the world knows the answer to that question and deals with it every night. They raise or lower the covers, put a leg outside the blankets, or do any of a number of things to adjust their sleep temperature.

Of course, all of this is in good fun. I doubt even the people—or at least most of them—who own these smart mattresses are taking this very seriously but it does make a point: we are—more and more—relying on a stable Internet connection for many aspects of our lives but that connection is extremely fragile. A large part of the problem is that a huge number of these services depend on Amazon, Google, or Microsoft to provide those services. That’s a lot of eggs in every few baskets.

Posted in General | Tagged , , | Leave a comment