Troubleshooting Mu4e Update

Recently, I wrote about my adventures in troubleshooting Mu4e. This is an update on what I discovered and what I did to resolve the problem. The chances are good that this won’t be of interest to anyone but I’m writing it down in case someone else has the same type of problems now or in the future.

The first thing I did was turn mu4e-index-cleanup off and mu4e-index-lazy-check on to speed up the reindexing. That turned out to be key to understanding what was going on. I noticed that with mu4e-index-cleanup off 2–3 thousand entries were being added to the store each time mu ran. That explained why mds (the spotlight indexer) was being kicked off each time mu ran. I turned mu4e-index-cleanup back on and the indexing was a bit faster than before I began with just mu4e-index-lazy-check on but mu was still running much longer than it should.

Next I noticed that mu.log in ~/.cache/mu/ had several entries like

2020-08-08 08:38:11 [WARN] cannot access /Users/jcs/Maildir/icloud/Trash/cur/1596883130.32552_36.Manfred:2,DS#T: No such file or directory

and that some entries in the Trash mailbox had impossible dates of 1969. There was obviously some problem in the Trash mailbox that was causing problems with the indexing.

I thought about deleting Maildir and downloading everything again from the IMAP server. There were a couple of problems with that, though. First it takes a long time and can be fussy. Second, I was worried that it might affect the renaming of posts and the links that point to them.

I finally just deleted the Trash mailbox, reinitialized the Xapian database, and ran index. That has apparently fixed all the problems. Mu4e is speedy again and even the CPU load has decreased from about 7% to 2%–3% when the system is idling.

I was lucky that the problems were in the Trash mailbox rather than in the one that holds all my saved emails. Regardless, if you have similar problems, I hope this post will help you track down what’s going on.

Posted in General | Tagged , | Leave a comment

A Proposal to Ban Adtech

A really great article by Gilad Edelman in Wired asks Why Don’t We Just Ban Targeted Advertising? The article is from March but I somehow missed it and it’s definitely worth writing about. Even without going into any details at all, almost everyone would agree that getting rid of targeted advertising and the malignant adtech industry that depends on it would increase the goodness in the world significantly.

But, it turns out, there are some specific beneficial outcomes that we could expect from getting rid of targeted ads. When you see the list

  • Privacy
  • Less hate and fallacious content
  • Less political manipulation and election interference
  • A possibly revitalized journalism industry
  • Less discrimination

it’s hard to believe that merely banning adtech would have a positive effect on all those areas but the article makes a pretty good case for it.

But what about the content providers? If we take away their advertising, wouldn’t all their content disappear? The article concludes no. Even Google—the chief adtech offender, along with Facebook—still earns 60% of its income from non-targeted ads and, of course, most serious research into the matter concludes that targeted ads aren’t really that much more effective (about 4% more).

Edelman concedes that banning adtech may be a “cure-some” rather than a cure-all but that it’s an excellent start in restoring the Internet to a better time before the advertisers starting surveilling everything we do and selling our most private activities to anyone willing to pay.

Posted in General | Tagged | Leave a comment

The Frame Patch

As I mentioned yesterday in my post about the just announced Emacs 27.1-RC2, Álvaro Ramírez left a comment to my post A Shortcut Manager for My Emacs-centric Workstation. In that post, I said that I was using yequake to pop up an Org-mode capture buffer from anywhere in the system. Jack Baty commented that yequake didn’t return to the previous application after yequake was terminated. I’d noticed that too and after a bit of research discovered that this is a macOS problem but that it could be (sort of) fixed by inhibiting Emacs from returning that the last frame when a new frame is deleted.

You can follow the discussion in the comments to the post but the TL;DR is that the fix involves a patch to frame.c. Since Emacs 27.1-RC2 had just been released, I decided to compile it with the patch and try things out. If you’d like to do the same thing, here’s the patch (against the RC2 release).

--- frame.c.orig	2020-08-07 13:54:20.000000000 -0400
+++ frame.c	2020-08-07 13:56:35.000000000 -0400
@@ -2056,15 +2056,6 @@
 		}
 	    }
 	}
-#ifdef NS_IMPL_COCOA
-      else
-	/* Under NS, there is no system mechanism for choosing a new
-	   window to get focus -- it is left to application code.
-	   So the portion of THIS application interfacing with NS
-	   needs to know about it.  We call Fraise_frame, but the
-	   purpose is really to transfer focus.  */
-	Fraise_frame (frame1);
-#endif
 
       do_switch_frame (frame1, 0, 1, Qnil);
       sf = SELECTED_FRAME ();

I said “(sort of) fixed” above because there are still two problems. First, I run Emacs and Safari in full screen mode. When I call yequake from one of them, it comes up in a different desktop. Safari doesn’t return to the Safari desktop. Oddly, with Emacs things work perfectly. The second problem is that even when yequake does return to the proper desktop, the previous application does not receive focus. Again, these problems are because of the way macOS works but the patch does help a lot.

It’s easy to try it out if you compile your own Emacs. If you get your Emacs from Homebrew, you can load the patched version with

brew install emacs-plus@27 --with-no-frame-refocus ...

Ramírez says that it should work with Emacs 26 too. Of course, if you’re not on macOS, you don’t have the problem at all.

Posted in General | Tagged | Leave a comment

Emacs 27.1-RC2

The Emacs 27.1 train keeps rolling along with the announcement of the second release candidate. I’ve been using 27.1-RC1 and haven’t encountered any problems so I wasn’t going to bother with RC2 but Álvaro Ramírez just explained to me how to fix the problem I and others are having with yequake. That involves patching frame.c so I’ll probably use RC2 when I do that.

There hasn’t been much discussion on Emacs-devel about RC1 problems so I expect that we’re close to an official release of 27.1.

Posted in General | Tagged | Leave a comment

Rolling Your Own Emacs Modal Mode

Lukas Lazarek has a very interesting post that details how he implemented his own evil-like mode for Emacs. Lazarek begins by describing the difference between Emacs and Vim. His distinction is much like mine: Vim wants to be a lightweight, fast editor and nothing else, while Emacs wants to be a text-based programming environment. I usually express that as Emacs being a lightweight Lisp machine but the meaning is pretty much the same.

Lazarek says the big advantage of Vim is its modal editing. I like modal editing but I see it as more something you’re used to rather than an actual advantage (unless we’re talking about RSI, which Lazarek does appear to be) but I don’t see it as an advantage. For me, the real advantage of Vim is not that it’s modal but that its command set is composable. I’ve written about this before.

Regardless, Lazarek wanted to implement a modal mode but didn’t want the exact Vim experience that Evil offers. It might seem like implementing your own Evil mode would be a significant undertaking but Lazarek demonstrates how simple it is by showing a short bit of Elisp that provides an “Evil mode” that merely implements the jklh cursor movement commands. It really is amazingly simple.

He also provides a link to his full implementation. It’s longer, of course, but still pretty simple and very easy to understand. That last clause is, I think, the main takeaway. It’s really easy to customize Emacs even in ways that seem complicated and outré. As always, Emacs lets you have it your way and often fairly easily.

Posted in General | Tagged | Leave a comment

Zettelkasten Explained and Demoed

Continuing with yesterday’s theme about Zettelkastens, today I’d like to point you at two videos that explain and demonstrate the idea. The first, ZETTELKASTEN METHOD (Explained Clearly with Examples and Software), by Mike and Matty, two brothers who are physicians among other things, explains what a Zettelkasten is, why you might find it advantageous to use one, and how to use it effectively. They demonstrate the ideas with Roam and show how notes can be linked together to make exploring your ideas and notes on a topic easy. The video is just over 6 minutes long so it should be easy to find time for it if you’re interested.

The second video, A Tour of Org Roam, is by Ian Jones. He gives a quick tour of Org-roam and demonstrates the Zettelkasten idea in an Emacs setting. He demonstrates the backlink feature and how it’s useful for organizing and finding notes. The best part of his video, I think, is the segment on Org-roam-server that presents a graphical representation of your notes and how they’re linked. You can click on a node and Org-roam will take you to the corresponding note. It’s a powerful feature and a great part of Org-roam. Jones’ video is just under 5 minutes so it will be even easier to find time for.

If you’re not an Emacser but are interested in the Zettelkasten concept, the first video will explain things without mentioning Emacs. If you are an Emacser, you should watch the second video too. It shows you an excellent implementation that’s available for you in Emacs.

Posted in General | Tagged , | Leave a comment

Zettelkasten Goes Mainstream

Until March of this year, I had never heard the term “Zettelkasten” or anything about the concept to which it applies. I first came across the idea in a post by Tiago Forte, which pointed me to Sönke Ahrens’ book, How to Take Smart Notes. I read the book and thought that it was a very powerful scheme that I could probably use in my workflow.

At the time, I felt like a pioneer. As far as I could tell, hardly anyone knew about the idea and virtually no one was writing about it. That’s all changed. Six months later, the idea is popping up everywhere and almost everyone concerned with personal information management is writing about it.

Over at Fast Company there’s an excellent article by Jack Beaudoin—Zapier on Zettelkastens and how to implement them. The article considers using existing applications, such as Evernote, OneNote, or Notion but concludes that these aren’t ideal solutions. In the first place, they fail to implement important Zettelkasten concepts such as atomicity and backlinks. Perhaps more importantly, they’re proprietary and offer no guarantees that they’ll be there for you tomorrow.

Another possibility is bespoke applications written specifically to implement the Zettelkasten idea. The premier example of this, of course, is Roam, which I’ve mentioned before. It’s a really excellent implementation of the Zettelkasten idea but, sadly, also suffers from its proprietary nature. The database is held on their servers so longevity is a real concern.

Finally, there are the solutions that use standard tools and the OS filesystem. These certainly speak to the proprietary and longevity issues but may not offer a good Zettelkasten implementation. They mention Obsidian but most of the ideas in this category aren’t really worthwhile. What they don’t mention, of course, are all the solutions available to Emacs users. That’s reasonable for media like Fast Company that aren’t addressed to geeks or the open source community. But if you are an Emacs user, you have your choice of the several solutions that have been discussed on Irreal recently. I continue to believe that Org-roam is the standout in this cohort but I still haven’t had a chance to experiment with it; I’m stuck struggling with mu4e.

Posted in General | Tagged , | Leave a comment

A Short Tutorial on ERT

Abrochard has an excellent short tutorial on using ERT for running regression tests on your Elisp code. Most of us, of course, don’t write enough Elisp to worry about regression tests but if you want to contribute to Emacs or an Emacs related project, it’s worth spending a couple of minutes learning about ERT.

It’s really easy to specify a test. In the simplest case you simply make one or more statements of the form

(should (equal <expected value> (<tested-function> <function-parameters>)))

and call ERT.

ERT can handle more complicated cases, including those that don’t return a value. Abrochard has a nice example of how to mock up a test when a function doesn’t return a value. Take a look at his post to see the particulars. There’s even more information in the ERT manual so take a look at that too. It’s really easy to use and there’s no reason not to take advantage of it.

Posted in General | Tagged | Leave a comment

Troubleshooting Mu4e

After I upgraded to macOS Catalina—or perhaps after I replaced my laptop because of the spilled drink catastrophe—I’ve been having problems with mu/mu4e. That’s serious for me because they are my primary email client. There are two problems.

First, mds and its helper apps were running not quite continually but regularly. Mds is the indexer for Spotlight, the macOS utility that provides system wide content searches. Mds uses a significant amount of CPU time when it’s running. So much so that the laptop gets significantly hotter. My hypothesis is that mds was getting kicked off every time mu ran mbsync to sync with the IMAP server—every 5 minutes on my machine. After it finishes running, there was a fairly short period until it got kicked off again.

I fixed that problem by telling Spotlight not to bother indexing my mail repository or the Xapian index that mu maintains. I can’t find anyone else complaining about this, which is odd because both my Spotlight configuration was the default (before I told it not to index the mail) and I’m not doing anything special in my mu4e configuration.

The second problem is sometimes it can take 30 seconds or longer for the email list to come up. Less often there is also a delay when I select a message before it appears. The delays appear to happen when mu is running. I haven’t figured out what’s causing this yet. I’d be happy to hear from anyone with some wisdom on the matter.

I’m inclined to think these problems were caused by the OS update and hope that they’ll be resolved when Big Sur is released. That happened before with a delay when invoking the browser from an Org link: the problem resolved itself with a new OS release. Again, if anyone has seen similar problems, let me know.

Posted in General | Tagged , | Leave a comment

Update On My 27.1RC1 Experience

It’s been a few days since I installed and starting using Emacs 27.1RC1 for all my day-to-day chores. If you follow the Emacs development list, all the items concerning 27.1 problems might lead you to think that it still wasn’t very stable but I haven’t found that to be the case at all.

By now, I’ve performed just about every one of the workflow operations that I do with Emacs and it’s performed flawlessly. The only difference I’ve seen is that Org-mode on large files is much speedier that it was before. It’s the same Org-mode so that probably means Emacs itself is faster.

I even tried EWW although I don’t usually use it much. It seemed about the same as before but I don’t have enough experience with it to say for sure. I’d really like to move as much browsing as possible into Emacs so perhaps I’ll try to use it more and get better familiar with it.

The TL;DR is that 27.1RC1 seems very stable to me, a very heavy user, and I have no hesitation in recommending that others upgrade to it. Doubtless the final version will be released soon so if you’re really wary just wait but as far as I can see there’s no reason not to start using it and help with the final testing. All you have to do if you find a problem is file a bug report, which, of course, you can do right from Emacs.

Posted in General | Tagged | Leave a comment