Emacs: The macOS Bug

4 days ago (xlii.space)

This is a problem with Emacs on virtually every GUI platform. Emacs insists that it owns the main loop, while most GUI frameworks insist that they own the main loop. Emacs wants to slurp some events from a queue-like thing, throw some drawing at another queue-like thing, then wait for another event. The GUI instead wants to call back into Emacs whenever an event comes in.

All that being said, Emacs has always worked pretty well for me on Mac. I use Emacs and PDFgrep to spelunk through multi-GBs of PDFs and it is faster than almost anything else.

  • As Daniel Colascione ('quotemstr' around these parts) said [1]:

    > GNU Emacs is an old-school C program emulating a 1980s Symbolics Lisp Machine emulating an old-fashioned Motif-style Xt toolkit emulating a 1970s text terminal emulating a 1960s teletype. Compiling Emacs is a challenge. Adding modern rendering features to the redisplay engine is a miracle.

    Emacs owns its main loop because, damnit, it created one before it was cool.

    Hats off to any heroes who would manage to drag it, kicking and screaming, into this millennium.

    [1] https://gist.github.com/ghosty141/c93f21d6cd476417d4a9814eb7...

  • I'm not sure if I understood the issue right, but if the issue is with the runloop, I think you can technically reimplement it yourself with nextEventMatchingMask.

    But their design just seems broken, if they're re-initializing the graphics context on every runloop iteration?

    • That’s what nsterm.m uses. As far as I can tell, the problem is that nextEventMatchingMask needs to be called from the NSApp main loop. But calling NSApplication:run blocks and takes over the main loop, which doesn’t work with for Emacs.

      So what Emacs does is keep control of its own main loop. It has a select() that listens for events, and then calls NSApplication:run in the event handler. Emacs’s implementation of run() processes all pending events and then exits and returns control to the real Emacs main loop. So every keystroke or timer event in Emacs invokes setting up and tearing down the entire NSApplication main loop.

      The relevant code is ns_select_1 in nsterm.m, line 5102. https://github.com/emacs-mirror/emacs/blob/master/src/nsterm...

      3 replies →

This is worth undertaking. macOS's stricter approach to handling some questionable hacks in Emacs could improve the codebase across all platforms. The PGTK frontend for Emacs (the Wayland-native frontend) was derived from the macOS version for instance. It replaced much of the messy X11 code with a cleaner, more modular Cairo-based frontend, which could be further enhanced by adopting a cross-platform, more future-proof SDL toolkit.

https://appetrosyan.github.io/posts/emacs-widget

Hopefully, similar improvements can address the issues with large locks and the lack of proper threading.

  • The problem with the PGTK frontend is it is notoriously EXTREMELY slow. The latency on user input compared to the X11 (especially Lucid) version has some people reverting back to X11/Lucid.

    When I do run Linux I run Wayland, I daily drive macOS, but better than both are what you already allude to: the Emacs widget toolkit which will focus on replacing the GUI frontend with SDL and also (equally potentially) introducing an actor-type framework (akin to BEAM's) for communication to decouple that GUI.

    • It's not just slow, it's also broken.

      Maybe broken is the wrong word, but it handles chained chords differently and it breaks my workflow at least.

      (C-s s C-s for instance needs you to release and press S again when lucid doesn't need it)

I recently noticed deleting a frame doesn't seem to free the associated memory; you can see this by running Activity Monitor and opening and deleting some frames. It's on the order of 10s of MB (exact amount depends on size of frame, whether it is fullscreen, etc). This is not much on a modern machine, but if you open and close lots of frames everyday (as I do) and keep Emacs running for weeks at a time (I do that too) then it starts to add up. My current kludge is to add a hook to resize a frame (which deallocate most of the frame memory) before deleting. This keeps the leaked memory to a level that is more tolerable.

(I've dug through the ObjC source, specifically "nsterm.m", but haven't quite figured out the core problem.)

[edited slightly for clarity]

  • Addendum: I wrote the original comment after skimming the article but before reading it (and the associated thread on emacs-devel) closely. These memory-related issues are (of course) what the article and the emacs-devel thread are about.

> It sounds horrible. But does it make sense? If it’s that bad - why didn’t anyone notice?

But they did! As you say yourself

> For many, this slowness won’t be a surprise. There are plenty of complaints about slowness on MacOS, especially around popular packages.

So this is some parody, describing as efficient something rather inefficient

> Because Emacs is very efficient.

> For instance, dragging a window handle - depending on the machine - could result in thousands if not millions of such events, causing allocation and reallocation of gigabytes of memory;

fwiw I live in [macOS emacs](https://emacsformacosx.com/) all day long for systems engineering (C/C++) and have 201 open buffers, an uptime of 57 days and ~540 MB memory usage.

  • I used that too, but have held back at 27.2 because all the following versions have been significantly slower.

    It seems around that time they added some processing loop that really made things sluggish.

  • I used to use that version of emacs, but performance issues on my Mac Studio made using it just untenable. I switched to Homebrew's "emacs-plus" which does not suffer, for whatever reason, the same performance issue. Based on TFA, I'm somewhat baffled as to why, but I can't argue with results.

there's another interesting issue on OSX which I've got a patch for in my homebrew version: on OS X, sleeping in a thread can hang, which causes LSP issues for me (through the `lsp-auto-install` package, since it downloads in a thread). The bug thread is interesting, but seems to have petered out; the patch works for me though!

[bug]: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=75275#83 [brew]: https://github.com/d12frosted/homebrew-emacs-plus/compare/ma...

  • Another example, if you run gunicorn with --preload on Macos, you need to export OBJC_DISABLE_INITIALIZE_FORK_SAFETY and some shenanigans.

> Even in the best case though, things won’t be as great as they are on Linux or Windows.

Worst case, you just swap out the NS p pselect hack and use a w32-like separate thread. Let Emacs be Emacs and let Cocoa be Cocoa.

NS would be just as good as Windows then. Isn't that bood enough? Maybe this thread splitting is the "deep event loop surgery" the author meant? I haven't been following.

On macOS and Linux, I haven't noticed any performance issues with Emacs. On Windows, however, the performance is significantly worse. To make matters worse, I even have to patch w32.c just to get it to build:

@@ -10298,7 +10298,7 @@ w32_read_registry (HKEY rootkey, Lisp_Object lkey, Lisp_Object lname) /* mingw.org's MinGW doesn't declare _dstbias. MinGW64 defines it as a macro. / #ifndef _dstbias -__MINGW_IMPORT int _dstbias; +__MINGW_IMPORT long _dstbias; #endif

/ Fix a bug in MS implementation of 'tzset'. This function should be

I do not seem to be experiencing this issue on my work machine, which is a 64 GiB MBP, running Homebrew's version of Emacs. It runs fine, but the endpoint security software activates every single time a binary is run, which means that Emacs workflows heavy on spawning subprocesses, like magit, tend to be slow.

Can't they just use Jemalloc with something similar to LD_PRELOAD under OSX?

Ah, ok, it's Cocoa related. It won't happen for instance with the Lucid build of Emacs for Mac OS X, but you might need an XServer for that. Altough I could be wrong and Lucid libraries can be run under OSX if they are ported...

And if someone manages to fix the bug described in the OP, he might have to maintain it as a fork because some influential emacs maintainers want it to be frustrating and unpleasant to use Emacs on non-Free OSes.

Emacs jank on macos has been slowly killing me. Enough so, that I am thinking of completely jumping ship after almost a decade of using emacs.

I often end up facing lag and performance issues in several different aspects of using emacs. Every time I boot up vim or any of the modern editors (zed/vscode), I get shocked at how smooth they are.

I only have 3 realistic options at this point:

- stop using macos (won't because macbooks are the best hardware I can get)

- stop using emacs

- keep suffering

currently I'm doing #3, but I soon need to make the hard call and swallow the pill.

What will my next editor be? Zed? NeoVim? write my own? Is there any other lisp/emacs like editor?

EDIT: helix looks cool

  • Just out of curiosity, what issue did you encounter? I have a quite customized emacs, and the only lag I really notice between Linux and macOS is in magit, where operations are noticeably slower.

    > What will my next editor be?

    Fancy giving a shot to Helix[0]? Not even is it pretty good out of the box, it has a scheme extension language in the work.

    [0] https://helix-editor.com/

    • Believe it or not but I tried to use Helix when my Emacs build was completely broken.

      It couldn't handle the Git repo of Emacs' size. Every keystroke took 3 seconds to process.

      I found it somewhat funny. But otherwise Helix is still a great editor worth recommending.

    • helix looks cool and the scheme PR open with STEEL looks quite at home for me. I'll check it out, Thanks!

      There is a plugin I can't live without: aggressive-indent, and it is awfully slow for me. I don't use any emacs distributions like doom, everything is hand rolled yet my keystrokes are noticeably slower than any other place.

      Sometimes random operations like projectile get slow down, sometimes I'm stuck hitting c-g multiple times, it keeps popping up every now and then.

      I need to restart emacs once every week because things tend to get slow by then.

      And yes, magit is the slowest of them all. I've spent weeks trying to debug and fix magit but it's so slow for me. I am a magit power user despite all the jank, because it really gives me superpower.

      Emacs has made me a much better developer, both because of repl driven development, and by making me grok how much power you can wield when you can mold an editor to your needs.

      Switching from emacs to something else will be a long and arduous journey for me, but I can't live with the jank anymore as I get frustrated by it almost every day.

      1 reply →

    • Have you tried installing the latest version of git via homebrew? I found it to be noticeably faster.

  • How about Emacs in the terminal? Now I'm a lowly neovim user so I don't know what I'm missing but I feel like I get on fine without a graphical web browser in my text editor or whatever Emacs people use the GUI mode for

  • There is fennel which compiles into lua. I know there are some people who use fennel almost exclusively, and have some sort of system set up that watches and auto-compiles and sources. I only ever used emac as a basic text editor in the terminal (years ago), so I can't say if this will be sufficient compared to the "real" experience in emacs. Just letting you know in case it is helpful.

    edit: I forgot to mention the most important thing, I am talking about using neovim

  • you might enjoy Lem! for ages i thought it was an editor _for_ common lisp, and then i learned the other day that it has built in lsp mode and highlighting for typescript and lots of other langs. it's pretty good!

    though i highly recommend writing your own editor. there aren't really any editors out there that can provide what emacs can provide someone who's been using it for almost a decade.

  • > Is there any other lisp/emacs like editor?

    There is Lem, an Emacs- like editor written in Common Lisp, which seems quite active.

  • Yeah I'm a 20 year emacs user in the same boat. Kinda curious what the learning curve is like going from emacs keybindings to vim style modal editing. Bonus points if Dvorak typists can tell me what they do because I've been sadly typing in Dvorak for even longer than I've been using emacs.

    • As a Dvorak typist and Emacs user (repented from using Vim in the past), when I need to use a vi-like I just use the standard key bindings. Nothing good comes from rebinding keys in my opinion. But the way my brain works is by remembering that scroll down in Vim is Control + D and my fingers remember where Control and D are located on the keyboard.

  • I was an Emacs user for many years. I used it to write my papers and dissertation (AUCTeX mode was great), and a huge amount of code.

    I switched to Vim, and later to NeoVim. I'd highly recommend it.

    It's scriptable, and these days is scriptable in multiple "real" programming languages. It took some getting used to, but I found myself going faster in vim than I ever did in Emacs.

    You might find https://vim-adventures.com/ fun for learning some of the basics. In particular, it's worth spending time learning the motion commands "in the small", because you'll spend a lot of time using them. For instance:

    t (up to character) and f (up to and including character)

    i( (inside parens, works with [ or < or ", or p for paragraph)

    a( (same thing but includes the delimiters).

    Things like that are extremely worth learning, in part because they're the "nouns" in vim's verb-noun editing model, so you'll use them in many different commands.

  • Even if macbooks were indeed the best hardware you can get, does having the best hardware you can get really matters more that having the best IDE you can get?

  • I liked emacs, still use it for a lot of things, but the instantly tinkering and changing got to me. Took longer to set something up to work how I wanted than to do the thing.

  • Try Doom Emacs with native-comp enabled - it significantly reduces the macOS jank while preserving your Emacs workflow and keybindings.

I think I will take some flak for this ... but honestly, I've always preferred running Emacs in my terminal. Weird things like this always happen when a windowing system gets involved (and I suppose, window system support was kind of bolted on to Emacs, not designed in from version 0). I started using the terminal exclusively on Linux when gtk + emacs --daemon was known to cause frequent crashes. I carried this over to using Mac OS without really thinking about it. I currently use Mac OS for work and some days I use Emacs with a native windows and sometimes just use my terminal. (Kitty is a really nice terminal. KKP is supported in Emacs with a third-party package, and kitty can do things like display underlines, italic, bold, etc. I use Windows Terminal at home and it really sucks compared to Kitty.)

There are advantages to using a native window (display your method doc popups in formatted markdown instead of plain text), but they have never really made up for the jank, so I've never committed to it. I'm glad the author took a look, though.

  • Some Emacs bindings don't work in the terminal, a big example is `C-M-%` , `query-replace-regexp` https://stackoverflow.com/questions/71302860/emacs-terminal-... to do a regex find and replace.

    • Yeah, that is one of the main reasons I don't use emacs in a terminal much (the main exception being that I use emacsclient with -t for terminal mode as $EDITOR and for quick editing). Some key bindings just don't work properly.

  • It depends on what you're trying to use it for. If you just use Emacs for coding, you're not going to gain anything from running it in a GUI. However, if you also use it to read documentation, take notes, etc.; being able to display images, different fonts, different sized fonts, etc. is pretty nice.

    • >If you just use Emacs for coding, you're not going to gain anything from running it in a GUI.

      That's true only if you think pointing devices are useless for editing and navigating code.

      ADDED: I withdraw this comment.

      8 replies →

  • I made the opposite choice with Vim, opting to always use GVim instead of the terminal. I was never able to set up terminal colors and mouse compatibility just right, and at 4k Gnome terminal would lag a bit. Terminals use windowing systems too. :P

    This was 10 years ago though, before Kitty and the new Windows Terminal.

  • Emacs in a terminal is the best.

    • The fact that Emacs is happy in the terminal is what keeps me in Emacs and away from GUI editors that come and go over the years - that and Emacs' longevity....

  • I started on vim and hopping back and forth between my editor and my terminal is just some muscle memory that I cannot break - so I too have always run emacs in the terminal ever since I made the switch.

  • I've been meaning to switch to GUI emacs for 15 years, I usually have it working, vterm is nifty... but it just works great in the terminal and with terminals getting SO FUCKING GREAT lately, eh, is it really an upgrade? WezTerm and ghostty on Wayland with no X jank? Accelerated, ligatures, mouse and keyboard clipboard integration, high-DPI...

    Can GUI anything keep up?

    • With a font like "Input Sans" you can fit about +30% more code on the same screen unless you're too pixel starved. Terminal can't do non-monospace. And even if it's just that I want character width to scale slightly with font weight...