My partner is an Astrophysicist who relies on Gnu Emacs as her daily driver. Her work involves managing a treasure trove of legacy code written in a variety of languages like Fortran, Matlab, IDL, and IRAF. This code is essential for her data reduction pipelines, supporting instruments across observatories such as Keck 1 & 2, the AAT, Gemini, and more.
Each time she acquires a new Mac, she embarks on a week-long odyssey to set up her computing environment from scratch. It's not because she enjoys it; rather, it's a necessity because the built-in migration assistant just doesn't cut it for her specialised needs.
While she currently wields the power of an M1 Max MacBook Pro and runs on the Monterey operating system, she tends to stick with the pre-installed OS for the lifespan of her hardware, which often spans several years. In her case, this could be another 2-3 years or even more before she retires the machine or hands it over to a postdoc or student.
But why does she avoid the annual OS upgrades? It's simple. About a decade ago, every OS update would wreak havoc on her meticulously set-up environment. Paths would break, software would malfunction, and libraries that used to reside in one place mysteriously migrated to another. The headache and disruptions were just not worth it.
She decided to call it quits on annual OS upgrades roughly 7-8 years ago. While I've suggested Docker as a potential solution, it still requires her to take on the role of administrator and caretaker, which, in her busy world of astrophysical research, can be quite the distraction.
When your partner builds her entire dev environment against a very specific version, packages etc. and then you expect it to just work next macOS Version? If you don't put any effort into using containers, vms or even just basic setup scripts then yeah this will not work out.
I've worked with a few physicists and they are scientists first and developers second. Which is okay, but will lead to janky setups that you cannot simply upgrade the OS under.
I want actual updates of my OS and don't want to be stuck forever on some specific version of openssh because some Astrophysicist decides to built her dev environment against it.
So either build a reproducible dev environment or don't complain that you cannot update without issues.
I am an academic mathematician, and I've noticed a huge culture difference between academia (at least in mathematics) and software development.
In the software world, it seems to be universally accepted that everything will change, all of the time, and one needs to keep up. If your dependencies break, it's your responsibility to update those too. Hence software "maintenance", which requires a lot of effort.
In my world, maintenance is not a concern. If I write a paper, then once it's done it's done. Academic papers, too, have dependencies: "By Theorem 3.4 of Jones and Smith [14], we can conclude that every quasiregular foozle is blahblah, and therefore..." You don't have to worry that Jones and Smith are going to rewrite their paper, change their definition of "quasiregular", or renumber everything.
What made you think the OP's post was a complaint? It seems reasonable to defer upgrades if you know they're going to break things and create a lot of extra admin work when you'd rather just be getting your job done.
I feel the same pain myself. Major macOS upgrades typically always break things, either in my dev environment or just by introducing bugs or breaking/removing/changing some random OS feature that I like using.
Its usually only Xcode which forces me to upgrade, since at some point in each release cycle Apple starts requiring the latest macOS for the latest Xcode, which in turn is required if you want to target/deploy to the latest iOS. If it wasn't for that I'd prefer to stick to one major version for 2-3 years.
While I can see some minor issues with version upgrades, some software simply have breaking changes, but mostly I'm surprised when people manages to create environments that are really locked to a single OS version.
It used to happen on Windows quite a bit, through now fault of Microsoft. People for one reason or another sometimes see to go out of their way to ensure that their software is tied entirely to one specific version of one operating system.
We dealt with a customer who could upgrade a server, required to use newer TLS versions, but their custom software was pretty much locked to Windows Server 2008. They couldn't even build the software on something newer. I quite honestly don't know how people keep ending up in that situation. I don't think they do it on purpose, but the way they insist on working just makes things work out that way.
There was a time when every macos upgrade broke VMware, guaranteed. I too am extremely conservative about OS upgrades. Why take the risk when there's almost no benefit? Let the early adopters iron out the bugs for you.
> and then you expect it to just work next macOS Version?
Yes I do. I couldn’t give a s how much security stuff or how many developer issues update solves, backwards compatibility is king. The older I get, the more I am about this. I have million things to take in my life, I’ve paid 3k for this thing and you expect me to sysadmin it on top?
This is OK for OS stuff but what do you do when xcode upgrade breaks how your apps are linked. This is what happened this time with erlang which uses wx for some of the apps. Now the poor erlang team has to backport fixes for all old versions as well.
You might suggest she write a MacPorts Portfile(s) (or the Homebrew equivalent) that describes the her setup. It is a distraction, but hopefully a one time distraction. It doesn’t have the overhead of running docker, gives her all the tools in well-known paths. IMHO (and as a fan), MacPorts has an advantage over HomeBrew by having all dependencies vendored so OS updates have less impact.
Edit to add, if you or her want help with this, I wouldn’t mind helping, reach out to my username at gmail.
> MacPorts has an advantage over HomeBrew by having all dependencies vendored so OS updates have less impact.
I use MacPorts, but be careful stating this. libc is unstable between major releases of Mac OS, and the recommendation from the MacPorts project itself is to reinstall *all* of your ports after an OS upgrade.[1] This is not a fun process if you don't delay updating until the build bots for the latest OS go online. Also note it is literally impossible to statically link libc in Mac OS applications.[2]
Realistically speaking, I think Nix is likely to be a better alternative to both MacPorts and Homebrew for setting up dev environments, since you will have a declarative file format rather than ad-hoc Tcl (or Ruby) scripts running a function for fetch, verify, configure, build, destroot, etc. The only reason I personally don't use Nix is because MacPorts has more or less just worked for me and I haven't had the motivation to learn how Nix really works.
Agreed. I switched from Homebrew to MacPorts a few years ago and couldn't be happier. I just delay upgrading macOS until MacPorts officially supports the latest OS (which often takes a bit longer than Homebrew).
NOTE: I briefly tried Gentoo Prefix, so I could use the same setup for both Linux and macOS, but that required quite a bit more time investment than I'm willing to deal with. I spent even less time trying out Nix, but the learning curve was steeper than I had the time for, so I gave up on it pretty quickly.
did you mean versioned? if you really meant vendored, could you elaborate?
I don't know about brew, but MacPorts wasn't ready when Sonoma came out. So that's a bit of a bummer for early adopter types, or folks that need to develop their own app against beta macOS and depend on ports for their setup.
Some unsolicited, anecdotal advice I hope will be appreciated -
After several years of perennial macOS environment hell (part of which was spent working in a much more research-oriented environment - e.g. lots of ancient HPC packages, etc.), I made the jump to just using Nix on macOS [0]. Takes a little bit of learning (realistically just a couple hours to get productive IME - just enough to get acquainted with nix-shell [1] and build some configs). After a few months, I had the thought to look at what I still used brew for and realized I could just move to Nix completely - and remove Brew. I back up all my nix configs to a git repo, just in case - and whenever I switch to a new machine, or "brick" my current one - I just reinstall nix, pull in my configs, and I'm good to go - takes 5 minutes (a conservative estimate tbh). The only caveat is to just check the community [2] before upgrading to the next macOS version to make sure any issues have been ironed out. In the early days of macOS support, it was a bit finnicky between updates - I haven't had any issues for the last couple years that weren't my fault (for example, booting into recovery mode and accidentally deleting the nix store APFS volume - even then, all I had to do was reinstall nix and pull my configs).
It is so nice to just "declare" I want to use and just...use it. Want to try out ripgrep for something?
`nix-shell -p ripgrep`
Not what you want? just exit the shell. Too much unused crap taking up space in your Nix store? `nix-collect-garbage`.
There's even darwin-nix [3] as a sort-of "nixos-for-macos" - I started using it recently, mostly for managing macOS settings declaratively, and it's great - but honestly 99% of the usefulness I get on macOS from Nix is just using the package manager!
I have the same exact recommendation. My work laptop was stolen and setting back everything was a matter of 10 mins. I have a custom emacs configuration, templated with nix, that only calls stuff from nix store, that I’m pretty much confident that I nothing will break as long as nix works.
Btw, I still use hombrew for some stuff I’m too lazy to create derivations, but I use nix-darwin homebrew module to also manage it with nix. The shitty part is that I must install homebrew. I think that can also be automated with a simple activation script, but I’m too lazy and it’s not something I do more than once on the machine.
Hey, I have a pretty extensive Emacs configuration [1] on my Macbook using Nix, nix-darwin, and the Nix community Emacs overlay. It's been stable across multiple OS updates, and if minor stuff breaks 99% of the time an issue is already open as it's broken for everyone. Really, Nix is pretty awesome for the dev environment use case; bore yourself through the syntax and be rewarded with an easily reproducible system.
Yeah, this is basically the perfect use case for nixpkgs. You can make sure the same versions are built every time and get configured exactly the same. Then home-manager can deal with the extra config files. No reliance on the system tools anymore beyond the core libraries and you're not bound by the brew update cadence either.
Came here to say the same thing. Nix can even configure Emacs with all packages, so that everything is static, deterministic, and reproducible. I am actually looking forward to the possibility of running NixOS on MacBooks.
- late adoption is a synonym to remain production ready. We used to have a badge in the internal system of Amazon called late adopter because most of the staff who run the amzn infra did not want to upgrade anything unless there was a must (mostly security fixes)
- setting up such environments by hand is a time wasting error prone excercise. Automation is a must. I keep an Ansible repo only for the environment I need to maintain.
- using MacOS for Fortran, Matlab etc might not be the most efficient. For problematic software I usually run a virtualized Linux instance that can be snapshotted and transferred to other devices so it is super easy to replicate a complicated environment
- Docker might not cut it, we live in an era when a fully virtualized instance boots up in 3 seconds on a M1 Mac so you might as well just use a full OS instead of a container
Btw. it is a shame that software development is like this. The lack of 1.0 mindset and the continuous "improvements" that are quite often regressions and UX degradations are really annoying and mostly unnecessary. There are tiny improvements between the last 3 major OS versions from Apple.
I could not care less about new notifications (that I immediately disable after installing the new MacOS) and such features and I care deeply about being able to run my usual tools as they were running in the previous version.
>- Docker might not cut it, we live in an era when a fully virtualized instance boots up in 3 seconds on a M1 Mac so you might as well just use a full OS instead of a container
This sounds like FUD to me, but then again you can't use docker with out a VM on OSX any way.
Setting apart the collection of specific, idiosyncratic application stacks in question here, it is frustrating to see how badly computer users are being served by the commercial operating systems. I expected APPL and MSFT to make more progress in software usability in general.
We are now in an age that I call "Abusability": How much can we insult and mistreat the customer with our unfriendly software? ^_^
> I've suggested Docker as a potential solution
I recommend Podman instead because it is compatible with Docker containers but does not require root privilege.[1]
> it still requires her to take on the role of administrator and caretaker, which, in her busy world of astrophysical research, can be quite the distraction
.
.
.
And yet
> she embarks on a week-long odyssey to set up her computing environment from scratch
She is already a sysadmin. But multiply the number of machines to maintain by a factor of 1000 for an initial model of a sysadmin's real workload.
I’ve found the Migration Assistant to be solid—almost fanatical—for persisting custom settings. Most recently, it carried over my non-standard OpenSSH + config with resident key support, GPG config, and a couple of other things without a hitch. I even found a few config files from 2007 and a same-era Safari SIMBL extension that probably stopped working in Snow Leopard, all faithfully migrated across five different machines and a dozen operating systems.
You know, it's probably not migration assistant. I would say it's a major OS upgrade and gets tarred with the same brush. I personally have found it to be fantastic for upgrading to new hardware.
The state of laptop and desktop computing is abysmal. I use Ubuntu as a daily driver for desktop and macOS for my laptop, but all three major OSes, including Windows, have many issues.
Gnome is relatively stable but not foolproof, there are strange behaviors like file picker previews not working. There are services running in the background that do who knows what. I have built a custom distribution using build root, I will be attempting to use that as my daily driver.
It might not be up to her requirements, but an Ansible playbook may help. It's what we use to set up new MacBooks and only runs in localhost mode.
Ansible is fairly quick to learn, and can control homebrew so it's able to install a lot of stuff.
If you're interested, ping me and I can share some snippets. Also I bet you'll get a reply from someone evangelizing Nix ;)
EDIT: Oh, and git repo with dot files is very useful. As is homebrew bundle, which installs a list generated from a different host.
EDIT 2: oh also I just moved MacBooks last week, so a lot of this fresh in my head right now. Including cargo/rust, VSCode profiles, brew bundle, ublock origin config, backups of .envrc files, etc etc ad infinitum. My Emacs config was actually about the easiest bit to move!
>Each time she acquires a new Mac, she embarks on a week-long odyssey to set up her computing environment from scratch. It's not because she enjoys it; rather, it's a necessity because the built-in migration assistant just doesn't cut it for her specialised needs.
She could use a bash or zsh script, with instructions to get the apps she wants. Brew allows installing most open source apps + MAS apps you've bought + third party proprietary apps. E.g.:
brew install --cask alfred
brew install --cask google-chrome
(She could save those in a brew dump file from the earlier machine too and re-play it).
then she could hookup up some external disk (or use some cloud storage) and sync any settings files she has customized (.zshrc, and so on) and rsync any stuff, build some folder structure she has, etc, and copy over settings for third party apps.
With "defaults write" commands she can then set a lof of Mac preferences that she'd normally have to change on the Settings page or individual Apple apps, e.g.:
When I was maintaining a setup script using brew a few years ago, it seemed to regularly break due to brew changing how to pin versions or something else. I just gave up and switched to Linux. Maybe brew is more stable these days.
Not really - if you were using python heavily you would not be using Apple python but installed from python.org or Anaconda or a package manager to control what version is in use
Also Apple did give an OS release or two warning I think.
It sounds like her use case is a good candidate for containerising the data reduction pipelines. If she can get the cruft to run in a container (or a few perhaps), she has instant access to get them running on a new computer as long as Emacs is "just" the frontend.
Stuff like Emacs is the easiest to set up. I've had my emacs config in git for years like most people. I've installed it on many different systems over the years. I also have all my dotfiles in another repo.
She should really just switch to GNU/Linux, though. So much easier.
this is the advice I've been hearing more often than not. I've been Linux and Windows for most of my career, and only strayed into Mac a few times in the last 5 years. Without fail, _every_ Mac I've owned has needed support involvement on an OS upgrade.
when I've told people that, what I hear is "you're not supposed to update the OS on a Mac, stick with the one it shipped with and never make the major OS update as those are for newer hardware".
to me that seems illogical, wrong, worse than Windows... and yet as an observer to comments like the parent and seeing colleagues get bitten by issues, it appears to be true... the happy path with Mac OS is to never upgrade the major version
I am not sure how to handle this post, I guess that's the same for RHEL or LTSes, if you upgrade major OSes you might break things, stay on current versions in order to stay safe, should we freeze software because astrophysicists are too busy?
A rolling Linux distro would introduce minor, easy-to-address changes every now and then. This may be less pain overall (much like making many small pull requests vs one giant rewrite).
Disclaimer: I'm running a rolling-release distro (Void) since 2016, and has gone through 5 machines with it, usually with near-zero setup after a move.
I don’t understand why they can’t suitably migrate things so they don’t break. They have kernel access to the system. It should be trivial to move whatever around that needs to be moved to carry on functionality.
I suspect it’s because they actually don’t know _what_ will break with what they change.
Nobody replicates work in these fields in that way.
Most of the time people just don’t replicate work, full stop. In the rare instances that they do, replicating the analysis from scratch is the whole point. Running the same script and getting the same result is not interesting.
While the macOS updates itself work quite well for me, e.g. I also didn't mind the 64 bit-only that much, anything relying on XCode command line tools is quite a pain. First I need to fight through the inconsistent warnings that the version is out-of-date/not installed and eventually find a Stackoverflow post which describes how to fix that. Afterwards a few other fixes are needed. I worked around it by relying more on Homebrew (which needs the occasional symlink fix) and limiting the number of tools I build myself. Still quite frustrating, I'll definitely switch back to Linux once my laptop won't get anymore security updates.
I realize Nix is also an option but the user land version is much more rudimentary than NixOS' - which already requires expert level knowledge of the package management for everyday chores.
To all who suggest nix or docker or whatever. Please consider that it should not be necessary in the first place. 20 years ago Apple advertised its laptops as "real Unix on your laptop" which rightly appealed to a lot of scientists
It is fair from the user to expect some stability here.
And it delivered - it is a real, as in "certified" unix. You are making here some assumptions about rate of updates, innovation and disruptions which have nothing to do with being a unix or unix-like and were never promised by apple as such.
I've done scientific work on a mac, and it was fine, now I am doing SWE work on a mac and it is fine, too. Such fragile setups would not survive most linux distro updates, the only advantage is that you could find a distro with a very low rate of innovation and thus a very long support window and call it day. But it is, obviously, not a viable choice for apple.
That's not a great idea security-wise. Apple has formally committed to only updating the latest major version for security fixes. Of course, they have backported some fixes, but even in early 2023 those backports were highly delayed and incomplete.
If you're talking 7-8 years (and the hardware is certainly good for that long) of still using the OS as shipped, I feel like that is a mistake. The most painful updates are when they do the very large refactors, like Mojave->Catalina, then Catalina->Bug Sir. I am guessing whatever comes after Sonoma will be a biggie also.
I am still on Catalina for my media server. I don't dare update it. But it's not connected to Internet.
Thinking of a mac as a fashionable media consumption device in this day and age seems odd to me. Maybe before the switch to arm, because honestly their intel devices were middling. But between the apple flavor of arm processor and amount of RAM (especially with the M3) these devices are serious work horses.
I used to daily drive a linux machine on a Thinkpad, and the experience just doesn't really compete imo, it would be hard to get me to switch back. It's not that it's a bad experience on a linux machine in the year 2023, but it's not without annoyances that add up over time that I just don't experience on mac.
For the record, the Dilbert comic[0] whence "here's a nickel, kid" mentions Unix, not Linux. macOS is a certified Unix.
Accordingly, Macs c/o Unix have been used to perform real work for decades.
Even so Don Norman, Apple Fellow, wrote the forward to the Unix Hater's Handbook. I say this to say there's serious thought behind Apple's OSes _pre-Unix_ and criticism of Unix. (See e.g. John Siracusa's encomia to the spatial Finder).
Can't speak for everyone, but for me... familiarity, hackability, simplicity, standards compliance.
GNU coreutils deviate from POSIX in a number of instances.
GNU coreutils are also verbose and potentially overengineered. Compare GNU ls.c (4726 LOC) to FreeBSD ls.c (1030 LOC), for example. Even if you include the other BSD source files used in ls, the FreeBSD version is still half the size of the GNU version. GNU grep is over 3000 LOC, whereas the FreeBSD version is 724.
I recently found out that Apple is shipping rsync 2.6.9. That's from 2006. It's missing a bunch of features, and I'm guessing a bunch of protocol improvements and security fixes...
It's because newer versions of rsync switched license to GPL3 (from GPL2), and apparently Apple is not touching that with a ten foot pole. It's also why the included bash is frozen in time (and probably why they switched to zsh as the default interactive shell)
Serious question, where is the effort going on each of these annual OS releases? I truly don't know, probably out of ignorance. Comparability with new HW? Supporting new file formats? As far as I can tell the headline feature of the latest MacOS is a screensaver with very large file sizes. Perhaps Apple and its many developers efforts would be better employed updating these essential tools and QA of the same.
A few years ago... I had stored some archives (more than ten years of personal stuff) directly under the file system root. A macos upgrade eliminated them. All of them. No warning in advance, no warning as it trashed my stuff. Really user-hostile.
Literally always. With Sonoma my laptop no longer goes to sleep and, uses a ton of CPU (and runs the fans at full blast) while the screensaver is on and along with about half of my colleagues, I am getting a massive amount of microphone echo from Google Meet when not using headphones where previously it was never an issue.
When I upgraded my MBA to Sonoma, it was unusable for about 3-4 days. Opening Finder would slow the system down to a halt and eventually crash.
I have no idea what the problem was, but it eventually sorted itself out after that (not before I was ready to wipe it all and start again, despite how uninterested I was in doing that). Search engines and GPT didn’t give me much - Spotlight reindexing was everyone’s best guess, but I saw no evidence of this.
The fact that I’d transitioned to putting all my dev work into iCloud didn’t help - I moved out of that pretty shortly after, iCloud really doesn’t like files changing that often.
The UX on macOS is still my favourite across all the OSs I’ve used for personal use. But when it stops working … it’s impossible to figure out why and impossible to do anything about it except wait and hope, or start over.
To make it more entertaining we should take bets on "what daemon will shit itself this release".
Right now I'm staring at the calendar daemon. Spending 70-80% CPU syncing a calendar that hasn't been in use for a few years. If I had screwed up that badly I'd be embarrassed and keen to ship a fixed version pronto. Apple doesn't seem to give a shit about software quality.
There's nothing real about that assumption. What almost certainly happened is people didn't use the -o option enough to notice. Even less so in a way that triggers the bug.
ah, you're right - it looks like it was part of ohmyzsh.
It appears I missed the -o; without it, my alias asserts, but unaliased doesn't. With the -o they both assert. Still not what I was expecting, but the failure to cleanly replicate it is mine.
Openssl is also "broken". As in if you generate a pfx export file using the openssl command included on macos you can not import the resulting pfx into the same machines Keychain.
My company uses Rippling with a security policy which forces you to install updates a few days after they are released (or it just closes all your shit without warning to install). Really hoping Sonoma doesn't break anything. So far I've never done a major OS update without spending at least an hour unfucking something that it broke.
My partner is an Astrophysicist who relies on Gnu Emacs as her daily driver. Her work involves managing a treasure trove of legacy code written in a variety of languages like Fortran, Matlab, IDL, and IRAF. This code is essential for her data reduction pipelines, supporting instruments across observatories such as Keck 1 & 2, the AAT, Gemini, and more.
Each time she acquires a new Mac, she embarks on a week-long odyssey to set up her computing environment from scratch. It's not because she enjoys it; rather, it's a necessity because the built-in migration assistant just doesn't cut it for her specialised needs.
While she currently wields the power of an M1 Max MacBook Pro and runs on the Monterey operating system, she tends to stick with the pre-installed OS for the lifespan of her hardware, which often spans several years. In her case, this could be another 2-3 years or even more before she retires the machine or hands it over to a postdoc or student.
But why does she avoid the annual OS upgrades? It's simple. About a decade ago, every OS update would wreak havoc on her meticulously set-up environment. Paths would break, software would malfunction, and libraries that used to reside in one place mysteriously migrated to another. The headache and disruptions were just not worth it.
She decided to call it quits on annual OS upgrades roughly 7-8 years ago. While I've suggested Docker as a potential solution, it still requires her to take on the role of administrator and caretaker, which, in her busy world of astrophysical research, can be quite the distraction.
This is really weird thing to complain about.
When your partner builds her entire dev environment against a very specific version, packages etc. and then you expect it to just work next macOS Version? If you don't put any effort into using containers, vms or even just basic setup scripts then yeah this will not work out.
I've worked with a few physicists and they are scientists first and developers second. Which is okay, but will lead to janky setups that you cannot simply upgrade the OS under.
I want actual updates of my OS and don't want to be stuck forever on some specific version of openssh because some Astrophysicist decides to built her dev environment against it.
So either build a reproducible dev environment or don't complain that you cannot update without issues.
I am an academic mathematician, and I've noticed a huge culture difference between academia (at least in mathematics) and software development.
In the software world, it seems to be universally accepted that everything will change, all of the time, and one needs to keep up. If your dependencies break, it's your responsibility to update those too. Hence software "maintenance", which requires a lot of effort.
In my world, maintenance is not a concern. If I write a paper, then once it's done it's done. Academic papers, too, have dependencies: "By Theorem 3.4 of Jones and Smith [14], we can conclude that every quasiregular foozle is blahblah, and therefore..." You don't have to worry that Jones and Smith are going to rewrite their paper, change their definition of "quasiregular", or renumber everything.
Different cultures. Personally, I prefer my own.
11 replies →
> "This is really weird thing to complain about."
What made you think the OP's post was a complaint? It seems reasonable to defer upgrades if you know they're going to break things and create a lot of extra admin work when you'd rather just be getting your job done.
I feel the same pain myself. Major macOS upgrades typically always break things, either in my dev environment or just by introducing bugs or breaking/removing/changing some random OS feature that I like using.
Its usually only Xcode which forces me to upgrade, since at some point in each release cycle Apple starts requiring the latest macOS for the latest Xcode, which in turn is required if you want to target/deploy to the latest iOS. If it wasn't for that I'd prefer to stick to one major version for 2-3 years.
6 replies →
While I can see some minor issues with version upgrades, some software simply have breaking changes, but mostly I'm surprised when people manages to create environments that are really locked to a single OS version.
It used to happen on Windows quite a bit, through now fault of Microsoft. People for one reason or another sometimes see to go out of their way to ensure that their software is tied entirely to one specific version of one operating system.
We dealt with a customer who could upgrade a server, required to use newer TLS versions, but their custom software was pretty much locked to Windows Server 2008. They couldn't even build the software on something newer. I quite honestly don't know how people keep ending up in that situation. I don't think they do it on purpose, but the way they insist on working just makes things work out that way.
7 replies →
There was a time when every macos upgrade broke VMware, guaranteed. I too am extremely conservative about OS upgrades. Why take the risk when there's almost no benefit? Let the early adopters iron out the bugs for you.
3 replies →
> and then you expect it to just work next macOS Version?
Yes I do. I couldn’t give a s how much security stuff or how many developer issues update solves, backwards compatibility is king. The older I get, the more I am about this. I have million things to take in my life, I’ve paid 3k for this thing and you expect me to sysadmin it on top?
This is OK for OS stuff but what do you do when xcode upgrade breaks how your apps are linked. This is what happened this time with erlang which uses wx for some of the apps. Now the poor erlang team has to backport fixes for all old versions as well.
[dead]
You might suggest she write a MacPorts Portfile(s) (or the Homebrew equivalent) that describes the her setup. It is a distraction, but hopefully a one time distraction. It doesn’t have the overhead of running docker, gives her all the tools in well-known paths. IMHO (and as a fan), MacPorts has an advantage over HomeBrew by having all dependencies vendored so OS updates have less impact.
Edit to add, if you or her want help with this, I wouldn’t mind helping, reach out to my username at gmail.
> MacPorts has an advantage over HomeBrew by having all dependencies vendored so OS updates have less impact.
I use MacPorts, but be careful stating this. libc is unstable between major releases of Mac OS, and the recommendation from the MacPorts project itself is to reinstall *all* of your ports after an OS upgrade.[1] This is not a fun process if you don't delay updating until the build bots for the latest OS go online. Also note it is literally impossible to statically link libc in Mac OS applications.[2]
Realistically speaking, I think Nix is likely to be a better alternative to both MacPorts and Homebrew for setting up dev environments, since you will have a declarative file format rather than ad-hoc Tcl (or Ruby) scripts running a function for fetch, verify, configure, build, destroot, etc. The only reason I personally don't use Nix is because MacPorts has more or less just worked for me and I haven't had the motivation to learn how Nix really works.
[1] https://trac.macports.org/wiki/Migration
[2] https://developer.apple.com/library/archive/qa/qa1118/_index...
23 replies →
Agreed. I switched from Homebrew to MacPorts a few years ago and couldn't be happier. I just delay upgrading macOS until MacPorts officially supports the latest OS (which often takes a bit longer than Homebrew).
NOTE: I briefly tried Gentoo Prefix, so I could use the same setup for both Linux and macOS, but that required quite a bit more time investment than I'm willing to deal with. I spent even less time trying out Nix, but the learning curve was steeper than I had the time for, so I gave up on it pretty quickly.
Cheers to this guy.
> dependencies vendored
did you mean versioned? if you really meant vendored, could you elaborate?
I don't know about brew, but MacPorts wasn't ready when Sonoma came out. So that's a bit of a bummer for early adopter types, or folks that need to develop their own app against beta macOS and depend on ports for their setup.
8 replies →
Some unsolicited, anecdotal advice I hope will be appreciated -
After several years of perennial macOS environment hell (part of which was spent working in a much more research-oriented environment - e.g. lots of ancient HPC packages, etc.), I made the jump to just using Nix on macOS [0]. Takes a little bit of learning (realistically just a couple hours to get productive IME - just enough to get acquainted with nix-shell [1] and build some configs). After a few months, I had the thought to look at what I still used brew for and realized I could just move to Nix completely - and remove Brew. I back up all my nix configs to a git repo, just in case - and whenever I switch to a new machine, or "brick" my current one - I just reinstall nix, pull in my configs, and I'm good to go - takes 5 minutes (a conservative estimate tbh). The only caveat is to just check the community [2] before upgrading to the next macOS version to make sure any issues have been ironed out. In the early days of macOS support, it was a bit finnicky between updates - I haven't had any issues for the last couple years that weren't my fault (for example, booting into recovery mode and accidentally deleting the nix store APFS volume - even then, all I had to do was reinstall nix and pull my configs).
It is so nice to just "declare" I want to use and just...use it. Want to try out ripgrep for something? `nix-shell -p ripgrep` Not what you want? just exit the shell. Too much unused crap taking up space in your Nix store? `nix-collect-garbage`.
There's even darwin-nix [3] as a sort-of "nixos-for-macos" - I started using it recently, mostly for managing macOS settings declaratively, and it's great - but honestly 99% of the usefulness I get on macOS from Nix is just using the package manager!
[0] https://nixos.org/download#nix-install-macos [1] https://nix.dev/tutorials/first-steps/declarative-shell [2] https://nixos.org/community/ [3] https://github.com/LnL7/nix-darwin
I have the same exact recommendation. My work laptop was stolen and setting back everything was a matter of 10 mins. I have a custom emacs configuration, templated with nix, that only calls stuff from nix store, that I’m pretty much confident that I nothing will break as long as nix works.
Btw, I still use hombrew for some stuff I’m too lazy to create derivations, but I use nix-darwin homebrew module to also manage it with nix. The shitty part is that I must install homebrew. I think that can also be automated with a simple activation script, but I’m too lazy and it’s not something I do more than once on the machine.
Hey, I have a pretty extensive Emacs configuration [1] on my Macbook using Nix, nix-darwin, and the Nix community Emacs overlay. It's been stable across multiple OS updates, and if minor stuff breaks 99% of the time an issue is already open as it's broken for everyone. Really, Nix is pretty awesome for the dev environment use case; bore yourself through the syntax and be rewarded with an easily reproducible system.
[1] https://github.com/dustinlyons/nixos-config
There are a few cases where Nix itself can break on macOS upgrades, but the recently released Survival Mode installer should mitigate that.
https://determinate.systems/posts/nix-survival-mode-on-macos
1 reply →
Yeah, this is basically the perfect use case for nixpkgs. You can make sure the same versions are built every time and get configured exactly the same. Then home-manager can deal with the extra config files. No reliance on the system tools anymore beyond the core libraries and you're not bound by the brew update cadence either.
Came here to say the same thing. Nix can even configure Emacs with all packages, so that everything is static, deterministic, and reproducible. I am actually looking forward to the possibility of running NixOS on MacBooks.
Couple of things here:
- late adoption is a synonym to remain production ready. We used to have a badge in the internal system of Amazon called late adopter because most of the staff who run the amzn infra did not want to upgrade anything unless there was a must (mostly security fixes)
- setting up such environments by hand is a time wasting error prone excercise. Automation is a must. I keep an Ansible repo only for the environment I need to maintain.
- using MacOS for Fortran, Matlab etc might not be the most efficient. For problematic software I usually run a virtualized Linux instance that can be snapshotted and transferred to other devices so it is super easy to replicate a complicated environment
- Docker might not cut it, we live in an era when a fully virtualized instance boots up in 3 seconds on a M1 Mac so you might as well just use a full OS instead of a container
Btw. it is a shame that software development is like this. The lack of 1.0 mindset and the continuous "improvements" that are quite often regressions and UX degradations are really annoying and mostly unnecessary. There are tiny improvements between the last 3 major OS versions from Apple.
I could not care less about new notifications (that I immediately disable after installing the new MacOS) and such features and I care deeply about being able to run my usual tools as they were running in the previous version.
>- Docker might not cut it, we live in an era when a fully virtualized instance boots up in 3 seconds on a M1 Mac so you might as well just use a full OS instead of a container
This sounds like FUD to me, but then again you can't use docker with out a VM on OSX any way.
2 replies →
Setting apart the collection of specific, idiosyncratic application stacks in question here, it is frustrating to see how badly computer users are being served by the commercial operating systems. I expected APPL and MSFT to make more progress in software usability in general.
We are now in an age that I call "Abusability": How much can we insult and mistreat the customer with our unfriendly software? ^_^
> I've suggested Docker as a potential solution
I recommend Podman instead because it is compatible with Docker containers but does not require root privilege.[1]
> it still requires her to take on the role of administrator and caretaker, which, in her busy world of astrophysical research, can be quite the distraction
. . . And yet
> she embarks on a week-long odyssey to set up her computing environment from scratch
She is already a sysadmin. But multiply the number of machines to maintain by a factor of 1000 for an initial model of a sysadmin's real workload.
= = =
[1] _ https://podman.io/
Maybe macOS is not the optimal platform for this kind of work (anymore).
I’ve relegated my Mac usage to simple web browsing and stuff like that.
Wouldn't it be simpler to just have a 'Nix running in a VMWare Fusion? Why go through this pain?
I am on a Mac, but my work is done on a Mint.
Native on apple silicon can be magical. I remember Jupyter being awesome once I found the goat to sacrifice to get it to install a few years back.
If it's already a massive house of cards, setting that all up all over again might be a nightmare.
Just run orbstack. Gives you nix or Debian. Or use a persistent docker.
I’ve found the Migration Assistant to be solid—almost fanatical—for persisting custom settings. Most recently, it carried over my non-standard OpenSSH + config with resident key support, GPG config, and a couple of other things without a hitch. I even found a few config files from 2007 and a same-era Safari SIMBL extension that probably stopped working in Snow Leopard, all faithfully migrated across five different machines and a dozen operating systems.
I’m curious what doesn’t work for her.
You know, it's probably not migration assistant. I would say it's a major OS upgrade and gets tarred with the same brush. I personally have found it to be fantastic for upgrading to new hardware.
The state of laptop and desktop computing is abysmal. I use Ubuntu as a daily driver for desktop and macOS for my laptop, but all three major OSes, including Windows, have many issues.
Gnome is relatively stable but not foolproof, there are strange behaviors like file picker previews not working. There are services running in the background that do who knows what. I have built a custom distribution using build root, I will be attempting to use that as my daily driver.
It might not be up to her requirements, but an Ansible playbook may help. It's what we use to set up new MacBooks and only runs in localhost mode.
Ansible is fairly quick to learn, and can control homebrew so it's able to install a lot of stuff.
If you're interested, ping me and I can share some snippets. Also I bet you'll get a reply from someone evangelizing Nix ;)
EDIT: Oh, and git repo with dot files is very useful. As is homebrew bundle, which installs a list generated from a different host.
EDIT 2: oh also I just moved MacBooks last week, so a lot of this fresh in my head right now. Including cargo/rust, VSCode profiles, brew bundle, ublock origin config, backups of .envrc files, etc etc ad infinitum. My Emacs config was actually about the easiest bit to move!
This sounds like a lot more work then a few extra button clicks?
5 replies →
The way Russians put it, "The hedgehogs cried as they prickled their skins but continued shagging the cactus".
had to look that one up:
https://www.reddit.com/r/russian/comments/32x2ml/what_does_м...
1 reply →
>Each time she acquires a new Mac, she embarks on a week-long odyssey to set up her computing environment from scratch. It's not because she enjoys it; rather, it's a necessity because the built-in migration assistant just doesn't cut it for her specialised needs.
She could use a bash or zsh script, with instructions to get the apps she wants. Brew allows installing most open source apps + MAS apps you've bought + third party proprietary apps. E.g.:
(She could save those in a brew dump file from the earlier machine too and re-play it).
then she could hookup up some external disk (or use some cloud storage) and sync any settings files she has customized (.zshrc, and so on) and rsync any stuff, build some folder structure she has, etc, and copy over settings for third party apps.
With "defaults write" commands she can then set a lof of Mac preferences that she'd normally have to change on the Settings page or individual Apple apps, e.g.:
A new system would just need some small time to run the script to get to taste. I've been using that for years and it works great.
When I was maintaining a setup script using brew a few years ago, it seemed to regularly break due to brew changing how to pin versions or something else. I just gave up and switched to Linux. Maybe brew is more stable these days.
1 reply →
Part of your story involves the point os update release where apple just straight up removed system python.
that would have been a nightmare
Not really - if you were using python heavily you would not be using Apple python but installed from python.org or Anaconda or a package manager to control what version is in use
Also Apple did give an OS release or two warning I think.
It sounds like her use case is a good candidate for containerising the data reduction pipelines. If she can get the cruft to run in a container (or a few perhaps), she has instant access to get them running on a new computer as long as Emacs is "just" the frontend.
Stuff like Emacs is the easiest to set up. I've had my emacs config in git for years like most people. I've installed it on many different systems over the years. I also have all my dotfiles in another repo.
She should really just switch to GNU/Linux, though. So much easier.
this is the advice I've been hearing more often than not. I've been Linux and Windows for most of my career, and only strayed into Mac a few times in the last 5 years. Without fail, _every_ Mac I've owned has needed support involvement on an OS upgrade.
when I've told people that, what I hear is "you're not supposed to update the OS on a Mac, stick with the one it shipped with and never make the major OS update as those are for newer hardware".
to me that seems illogical, wrong, worse than Windows... and yet as an observer to comments like the parent and seeing colleagues get bitten by issues, it appears to be true... the happy path with Mac OS is to never upgrade the major version
The happy path with macOS is to only use hardware and apps made by Apple, I think that’s the problem.
If you stick to that, upgrades are usually pretty safe.
2 replies →
I am not sure how to handle this post, I guess that's the same for RHEL or LTSes, if you upgrade major OSes you might break things, stay on current versions in order to stay safe, should we freeze software because astrophysicists are too busy?
A rolling Linux distro would introduce minor, easy-to-address changes every now and then. This may be less pain overall (much like making many small pull requests vs one giant rewrite).
Disclaimer: I'm running a rolling-release distro (Void) since 2016, and has gone through 5 machines with it, usually with near-zero setup after a move.
4 replies →
I don’t understand why they can’t suitably migrate things so they don’t break. They have kernel access to the system. It should be trivial to move whatever around that needs to be moved to carry on functionality.
I suspect it’s because they actually don’t know _what_ will break with what they change.
1 reply →
With the difference that major LTS upgrades happen much less frequently than major macOS upgrades.
Nix-darwin seems like a solution worth looking into for this.
Ideally she would have the dependencies written down. How else are people supposed to replicate her work?
Nobody replicates work in these fields in that way.
Most of the time people just don’t replicate work, full stop. In the rare instances that they do, replicating the analysis from scratch is the whole point. Running the same script and getting the same result is not interesting.
While the macOS updates itself work quite well for me, e.g. I also didn't mind the 64 bit-only that much, anything relying on XCode command line tools is quite a pain. First I need to fight through the inconsistent warnings that the version is out-of-date/not installed and eventually find a Stackoverflow post which describes how to fix that. Afterwards a few other fixes are needed. I worked around it by relying more on Homebrew (which needs the occasional symlink fix) and limiting the number of tools I build myself. Still quite frustrating, I'll definitely switch back to Linux once my laptop won't get anymore security updates.
I realize Nix is also an option but the user land version is much more rudimentary than NixOS' - which already requires expert level knowledge of the package management for everyday chores.
To all who suggest nix or docker or whatever. Please consider that it should not be necessary in the first place. 20 years ago Apple advertised its laptops as "real Unix on your laptop" which rightly appealed to a lot of scientists
It is fair from the user to expect some stability here.
And it delivered - it is a real, as in "certified" unix. You are making here some assumptions about rate of updates, innovation and disruptions which have nothing to do with being a unix or unix-like and were never promised by apple as such.
I've done scientific work on a mac, and it was fine, now I am doing SWE work on a mac and it is fine, too. Such fragile setups would not survive most linux distro updates, the only advantage is that you could find a distro with a very low rate of innovation and thus a very long support window and call it day. But it is, obviously, not a viable choice for apple.
4 replies →
I have slowly moved to macos as my "front end" to all the other machines.
I, too, decided to skip on macOS upgrades for the lifespan of the hardware. But I tend to buy a Mac every 2-3 years so it's just a few skips.
She looks to be the perfect candidate for RedHat Enterprise Linux instead.
You mean Rocky Linux, no?
2 replies →
> she tends to stick with the pre-installed OS
That's not a great idea security-wise. Apple has formally committed to only updating the latest major version for security fixes. Of course, they have backported some fixes, but even in early 2023 those backports were highly delayed and incomplete.
If you're talking 7-8 years (and the hardware is certainly good for that long) of still using the OS as shipped, I feel like that is a mistake. The most painful updates are when they do the very large refactors, like Mojave->Catalina, then Catalina->Bug Sir. I am guessing whatever comes after Sonoma will be a biggie also.
I am still on Catalina for my media server. I don't dare update it. But it's not connected to Internet.
> That's not a great idea security-wise.
Unfortunately, a computer that doesn't do whatever you need it to do is useless, regardless of how "secure" it is.
1 reply →
>Catalina->Bug Sir
Was that intentional or freudian?
Just use Nix.
I have my entire Mac setup in a single set of declarative files, including custom compile of Emacs ans all of my config.
It takes maybe 20 minutes to have a new Mac up and running. 10 if on fast internet.
I have a Settings folder for this. It includes a Brewfile and a lot of other configs. It cut down that setup time to under a day
Sounds like Apple is cribbing from the Debian playbook.
[flagged]
[flagged]
Thinking of a mac as a fashionable media consumption device in this day and age seems odd to me. Maybe before the switch to arm, because honestly their intel devices were middling. But between the apple flavor of arm processor and amount of RAM (especially with the M3) these devices are serious work horses.
I used to daily drive a linux machine on a Thinkpad, and the experience just doesn't really compete imo, it would be hard to get me to switch back. It's not that it's a bad experience on a linux machine in the year 2023, but it's not without annoyances that add up over time that I just don't experience on mac.
1 reply →
For the record, the Dilbert comic[0] whence "here's a nickel, kid" mentions Unix, not Linux. macOS is a certified Unix.
Accordingly, Macs c/o Unix have been used to perform real work for decades.
Even so Don Norman, Apple Fellow, wrote the forward to the Unix Hater's Handbook. I say this to say there's serious thought behind Apple's OSes _pre-Unix_ and criticism of Unix. (See e.g. John Siracusa's encomia to the spatial Finder).
[0] https://i.imgur.com/z96dZ0x.jpg
1 reply →
Switching to bsd grep is one of the first things I do on a new macOS install. The builtins are old as hell!
I just install GNU grep and add it to PATH so I get the same version as on Linux:
https://formulae.brew.sh/formula/grep
Same, but BSD feel is also old as hell. Ripgrep all the way.
rg is massive “The UNIX programming environment” vibes.
Lightning fast.
Sane defaults.
Obviously rust too.
Most used tool on my computer.
2 replies →
Any reason for picking the BSD variants of usedland tooling over the GNU ones?
Can't speak for everyone, but for me... familiarity, hackability, simplicity, standards compliance.
GNU coreutils deviate from POSIX in a number of instances.
GNU coreutils are also verbose and potentially overengineered. Compare GNU ls.c (4726 LOC) to FreeBSD ls.c (1030 LOC), for example. Even if you include the other BSD source files used in ls, the FreeBSD version is still half the size of the GNU version. GNU grep is over 3000 LOC, whereas the FreeBSD version is 724.
16 replies →
That article is about FreeBSD's grep.
Ofc I meant GNU grep, not BSD grep. My bad!
I can understand shipping the old stuff because you don't give a fuck, but why on earth are they .. patching them. Jesus.
The explicit stated reason is to avoid copyleft.
4 replies →
In one of the comments:
Oh my!
FTR it's about en_GB.utf8 vs en_GB.UTF-8. For a moment I thought it's about lang vs LANG which would be expected behaviour IMO.
Let’s not forget that macOS BSD userland utilities are also super old and outdated.
I recently found out that Apple is shipping rsync 2.6.9. That's from 2006. It's missing a bunch of features, and I'm guessing a bunch of protocol improvements and security fixes...
It's because newer versions of rsync switched license to GPL3 (from GPL2), and apparently Apple is not touching that with a ten foot pole. It's also why the included bash is frozen in time (and probably why they switched to zsh as the default interactive shell)
Serious question, where is the effort going on each of these annual OS releases? I truly don't know, probably out of ignorance. Comparability with new HW? Supporting new file formats? As far as I can tell the headline feature of the latest MacOS is a screensaver with very large file sizes. Perhaps Apple and its many developers efforts would be better employed updating these essential tools and QA of the same.
There’s tons of things that are new in every OS version
The consumer facing site shows consumer facing features. Hence why you’d see things like screensavers and video effects highlighted.
The developer side features are documented on videos released at WWDC each year. https://developer.apple.com/news/?id=cj1fjit2
Or summarized in the Platform State of the Union https://developer.apple.com/news/?id=gxtoci7w
1 reply →
Good enough for UNIX certification, though.
I dread every MacOS update. It's always so painful, you never know what they're going to break.
A few years ago... I had stored some archives (more than ten years of personal stuff) directly under the file system root. A macos upgrade eliminated them. All of them. No warning in advance, no warning as it trashed my stuff. Really user-hostile.
You're saying you store personal stuff right at '/'? Apple did give warnings it was going to push forward with securing the rootfs.
2 replies →
I really dislike these kinds of comments. You don't give any example, but you say "always painful"? Literally always?
What exactly broke for you in the last three updates?
Literally always. With Sonoma my laptop no longer goes to sleep and, uses a ton of CPU (and runs the fans at full blast) while the screensaver is on and along with about half of my colleagues, I am getting a massive amount of microphone echo from Google Meet when not using headphones where previously it was never an issue.
When I upgraded my MBA to Sonoma, it was unusable for about 3-4 days. Opening Finder would slow the system down to a halt and eventually crash.
I have no idea what the problem was, but it eventually sorted itself out after that (not before I was ready to wipe it all and start again, despite how uninterested I was in doing that). Search engines and GPT didn’t give me much - Spotlight reindexing was everyone’s best guess, but I saw no evidence of this.
The fact that I’d transitioned to putting all my dev work into iCloud didn’t help - I moved out of that pretty shortly after, iCloud really doesn’t like files changing that often.
The UX on macOS is still my favourite across all the OSs I’ve used for personal use. But when it stops working … it’s impossible to figure out why and impossible to do anything about it except wait and hope, or start over.
To make it more entertaining we should take bets on "what daemon will shit itself this release".
Right now I'm staring at the calendar daemon. Spending 70-80% CPU syncing a calendar that hasn't been in use for a few years. If I had screwed up that badly I'd be embarrassed and keen to ship a fixed version pronto. Apple doesn't seem to give a shit about software quality.
Apple introduced a bug in grep that, under certain circumstances, causes grep to assert.
The real news here is not that Grep got broken…
The real news is that people who work at apple don’t use grep.
(Or persistent ssh connections…)
(Or multiple monitors…)
There's nothing real about that assumption. What almost certainly happened is people didn't use the -o option enough to notice. Even less so in a way that triggers the bug.
Interesting - it only appears to affect me with --color set (which is a default alias, but if you unalias grep it stops exploding.)
Your alias is not part of the macOS default.
ah, you're right - it looks like it was part of ohmyzsh.
It appears I missed the -o; without it, my alias asserts, but unaliased doesn't. With the -o they both assert. Still not what I was expecting, but the failure to cleanly replicate it is mine.
1 reply →
lol, I'm in this file.
As tedu, or what do you mean?
Aye, though I checked the rev number and they apparently forked from before I added -o support, so I'm in the clear.
2 replies →
https://github.com/apple-oss-distributions/text_cmds/blob/c0...
1 reply →
Openssl is also "broken". As in if you generate a pfx export file using the openssl command included on macos you can not import the resulting pfx into the same machines Keychain.
You have to use the legacy flag!
And you have to use -passout for the -passin, or -noenc.
Yeah, grep '.' was broken in previous versions (maybe it's a zsh builtin, but it is broken), it would give you repeated lines
My company uses Rippling with a security policy which forces you to install updates a few days after they are released (or it just closes all your shit without warning to install). Really hoping Sonoma doesn't break anything. So far I've never done a major OS update without spending at least an hour unfucking something that it broke.
Looks like the test suite isn't comprehensive enough to currently make it possible to work on that grep implementation with confidence.
I immediately switched to GNU grep, I hate the built in Mac/BSD utils. And now i've completely switched to ripgrep <3
Anyone having better luck with a clean install vs upgrading to Sonama?
I honestly feel like the fix here is replacing the assert with advance = advance > 0 ? advance : 1;