Useful built-in macOS command-line utilities

2 years ago (weiyen.net)

A couple more:

    afconvert(1) - an audio file format converter, which includes Apple's superior AAC codec from the Core Audio framework

    diskutil(8) - tons of tools for fixed and removable storage

Examples:

    afconvert in.wav -o out.m4a -q 127 -s 2 -b 160000 -f m4af -d 'aac '

    mb=300; diskutil eraseVolume APFS myramdisk `hdiutil attach -nomount ram://$((mb*2048))`

  • Mostly I use XLD (https://tmkk.undo.jp/xld/index_e.html) for audio conversion (as I'm mostly converting from .BIN + .CUE to "iTunes Plus" AAC for uploading to iTunes Match); but my understanding is that under the covers it's mostly just using afconvert (or whatever the system-framework equivalent of it is.)

    So if your needs are just "one audio file in, one audio file out, and let me tell you exactly what it should look like", then afconvert is probably what you want.

    • XLD is great. The best thing about it is the heavy parallelization if you have many cores to throw at the problem. You can convert mountains of albums very fast.

      Love those type of OG Mac indie software.

    • Yes, iTunes also uses the same Core Audio framework. In fact, even iTunes on Windows uses the same code base through the Quicktime-for-Windows libraries it runs off of.

      I prefer to encode with afconvert on the command line because it gives me a few more options for tweaking things, that I don't have access to in iTunes (or "Music" as it's called these days). Additionally, I use a simple shell script that handles all of it when for example ripping a whole CD.

    • +1 to XLD. I have been using it for years, it's a wonderful piece of software.

  • > afconvert

    Oh wow! A while back I ripped some concert audio from Youtube, but it was too large for me to sync using my `iTunes Match`. I've been too lazy to figure out how to downsample it juuuuust enough. But it looks like this works right out of the box

  • "superior" ... It looks silly to include such a judging adjective in the description of a comman line utility. I don't need "subtle" Apple ads on the command line. And aac is about the last format I would choose, encoding my audio in.

    • To be very clear, the man page is claiming that Apple's implementation of the AAC codec [the encoder specifically] is superior to other implementations of AAC encoding; not that AAC is superior to other audio codecs.

      If you're wondering how one implementation of an audio codec could be superior to others — mostly, it's because any lossy audio codec has an encoding phase called "psychoacoustic compression", where each implementation of the codec is free to do whatever it likes to "simplify" the waveform in some way (most easily pictured: by taking a Discrete Cosine Transform of the waveform, and then quantizing / convoluting some parts of the frequency space. Like what JPEG does to discard information from the chroma channels.)

      IIRC, rather than blunt-force quantization, Apple's AAC encoding does clever things (akin to the instrument separation done to audio in Melodyne) to split the waveform into "features", and then discards information in such a way that the features' separability is maintained (i.e. it doesn't become harder to "pick out" any given "sound" from the audio.)

      1 reply →

    • I think those are the commenter's descriptions, not Apple's man page synopses. My system has

           afconvert -- Audio File Convert
           diskutil -- Modify, verify and repair local disks.

    • Its quality is technically superior as evident from the product's PSNR and other factors. There are no "ads" here, I'm not a brand warrior, and nobody cares what formats are the last ones you'd encode your audio in.

  • There's also afplay - Audio File Play (plays an audio file to the default audio output). Can play all audio formats supported by CoreAudio.

  • How does that compare with ffmpeg? The arguments seem about the same.

    • afcovert uses the superior inbuilt AAC converter. FFmpeg can do this as well with the right arguments but you have to dig them up and the quality is capped to a lower value than you can get with afconvert.

      7 replies →

    • ffmpeg offers a few different encoders for many of its supported audio/video formats, and the result depends on what encoder you tell ffmpeg to use. It does not have support for using Core Audio on macOS but it does offer FDK AAC, which is one of the better AAC encoders available today - though not nearly as good as the one available in Core Audio.

      11 replies →

Few additions.

open -n file.pdf : opens new instance of Preview application which is useful if you want to open the same file twice (for example to look at different pages at once).

caffeinate -d : prevents display turning off, useful if you want to look at display without moving mouse.

  • Another useful caffeinate tip is the `-w` option.

    You can use it to pass a pid to keep the computer awake until that process completes. I use it for longer-running scripts that I don't want interrupted

  • I use this all the time:

    open -a <GUI Application> <File>

    Handy for distinguishing between editing and consuming media.

  • open -a is nice, i use it with alias, ex:

      alias qt='open -a "quicktime player"'
      alias vlc='open -a "vlc"'

  • `caffeinate -disu` is the best combination (that is, enable all options): your laptop won't go to sleep, won't dim the screem etc.

  • open -a "Finder" . - open Finder in the current directory.

    Standard apps usually just need the name, like Finder and Safari but you can also specify the path "/Applications/DifferentFinder.app"

    • Finder is pretty good, and it's handy to be able to open it from the terminal. But I find it super annoying it litters everything with .DS_Store files and there is no way to turn that off, except for external and network drives. Aside from, obviously, using a different file manager. Very un-Apple.

      7 replies →

    • Don't forget about "open -b <bundle_id> foo.txt" for example,

          open -b com.sublimetext.4 file.txt
      

      I prefer this vs using the App Name or Path, since those can vary depending on the system or app version. The bundle ID is more robust.

    • you can just

          open .
      

      unless you've reconfigured something else to open directories, which most people haven't.

  • That's one of macOS pitfalls: the inability to open 2 instances of the same app, simply using the default GUI is annoying sometimes.

    I think Windows is right in that matter...

  • Does `open` give focus? It used to, but since a few releases ago the app opens in the background, which is pretty annoying.

    My poor workaround is to use osascript: `tell application "System Events" to set frontmost of process "Finder" to true`

    • Isn’t open opening apps in the background a consequence of having “secure keyboard entry” enabled in Terminal.app?

  • Also, `cd .` to open the current directory in a Finder window.

    Not mine (found online years ago), but here's the opposite. `cd` into the frontmost Finder window:

      cd "$(osascript -e 'tell app "Finder" to POSIX path of (insertion location as alias)')";

I would like to also recommend the app:

   hear (macOS speech recognition and dictation via the command line)

See: https://sveinbjorn.org/hear

(Uses built-in macOS capabilities for transcription from audio to text.)

The terminal version of Disk Utility is actually much better than the GUI (it doesn't hang and the app is glitchy.

Docs are at https://ss64.com/mac/diskutil.html

  • Disk Utility used to be excellent, a model of how an app should be. But then they rewrote it in Swift and now it's just bad.

    Apple promotes Swift heavily but the results are not really encouraging. I don't think the "so-so" results are entirely because of Swift (probably due to newer, less battle tested software and also newer/younger devs) but still the fact is, all the not-so-great new software from Apple came with Swift rewrites, hard to not make a connection...

    • > all the not-so-great new software from Apple came with Swift rewrites, hard to not make a connection...

      That's a leap, but I understand your point of course. It _MIGHT_ also be said it all came from ObjectiveC experts having a first go at Swift, too. Or from groups that were run by some common set of "get it out the door" middle management types.

      Or any of a thousand reasons.

      1 reply →

If you want the least useful macOS commandline utility, 'pdisk' is:

     "...a menu driven program which partitions disks using the standard
     Apple disk partitioning scheme described in "Inside Macintosh: Devices".
     It does not support the Intel/DOS partitioning scheme[.]"

Although not built in, we have a list of easy to install command-line utilities for macOS.

https://terminaltrove.com/categories/macos/

You might find one you've never heard of that is useful! :)

  • Kinda related: does anyone else regularly finds CLI tools, installs them and never ever even call them once?

    • That's the difference between recall memory and recognition memory. The GUI took off because we can recognize a menu and easily figure out where we need to go, versus having to memorize obscure and cryptic commands. Honestly having a LLM spit out command line arguments is probably the best way forward if things don't have a GUI.

      1 reply →

    • Yes, been there, forgot that. I have since created a shell helper function that prints a list of "new and cool" CLI tools that I recently added to my dotfiles setup, which helps me committing them to long-term memory.

    • It happens but Ctrl-R has helped me get much better at finding commands I’ve used even just once before based on some tiny thing I remember about them.

    • Ha, when I search through my history I usually find I ran them exactly once, right after installing, and then forgot all about them. But there are some exceptions, I often use httpie instead of curl, MTR is great, I am liking doggo instead of dig...

  • This looks like a list of cross-platform command line tools. What about it is Mac-specific?

It's probably been written about a lot of places already.

For me, increasing the number of icons in the launcher grid was very useful.

After running these three commands, the size of the Launcher will be set to 13x8 apps:

    defaults write com.apple.dock springboard-columns -int 13
    defaults write com.apple.dock springboard-rows -int 8
    defaults write com.apple.dock ResetLaunchPad -bool TRUE; killall Dock

In general, a lot of parameters of different applications can be changed via command `defaults`

https://macos-defaults.com/

To get a complete list of parameters, you can execute

    defaults read

$ say Hello

To scare your teammates when you are logged in remotely optionally with

$ osascript -e "set volume output volume 100"

Not a command, but a little known feature of the Terminal app:

(shift+command+K) or Menu 'Shell' -> 'New Remote Connection...'

opens a SSH, S(FTP), TELNET connection manager window!

  • And sometimes handy: shift+cmd+. to toggle showing hidden (dot) files in Finder.

  • Sort of like command+k in the Finder, connects to a server. You can type in vnc://host or vnc://localhost:port... the latter is for VNC to hosts via an SSH tunnel.

    It's quite a good VNC client, too.

As they seem to have removed Bluetooth Explorer and all ways to get diagnostic info about the bluetooth system and/or change codecs and settings, does anybody know any good cmdline ways in later mac osxes to do the same?

For example I'm having a problem that comes and goes now and then where Bluetooth audio is 300 ms delayed compared to the video playback everywhere except in Youtube on Safari, very strange. It's good for a few months then suddenly it becomes unusable, then back to zero sync delay after a few months.

I was thinking this might be related to CODEC selections etc or some hidden setting that might get changed which we normally aren't allowed to determine :)

(btw I know there is a difference between latency and synchronization - latency might be unavoidable but video sync should always be able to compensate - I got curious on how exactly that works, where in the app / SDK / OS pipeline does the a/v sync happen on a Mac?)

More:

  nc(1) - netcat, arbitrary TCP and UDP connections and listens

  networkQuality - Speed test + network stress tool.

  system_profiler(8) - Useful way to grab extensive system information in shell scripts.

  wdutil(8) - wdutil provides functionality of the Wireless Diagnostics application in
     command line form.

One-liner for previewing a file with Quick Look. I aliased this to `ql` :)

    qlmanage -p $argv >/dev/null 2>&1

It misses the most important of them all, if you are used to copy content to usb drive for reading on a multimedia player : dot_clean -m

To find what causes your laptop drains its battery, you can use

    sudo powermetrics

  • Thanks Spotify:

    coreaudiod is using very high CPU at 111.90 ms/s

    I'm on a 16" M1 Macbook Pro 16 gig.

    • If spotify use the coreaudio daemon for the decoding it can be spotify's fault for this CPU usage, don't you think ?

      Maybe they are using it "wrong" but Apple Music isn't exactly light on ressource either...

I'll add `plutil` to the list. It's great for reading plist files, but did you know it can parse json too?

/usr/bin/plutil -extract your.key.path raw -o - - <<< "$jsoninput"

(obviously, less useful now that `jq`is built in)

  •     ; which jq
        /usr/bin/jq
        ; jq
        jq - commandline JSON processor [version 1.6-159-apple-gcff5336-dirty]
    

    Wow. When did `jq` start shipping by default? TIL

sips https://ss64.com/mac/sips.html

  • I have thousands of old photos that preview can open, but I can’t upload them into the photo.app because the file format is wrong. I’m going to try and use SIPS to convert them all into png first and then add to photo.app. Thanks for this pointer.

    • Similarly, I've had trouble getting audio files into a format that the Books app understands (for ebooks), until I found

         afconvert -v -s 3 -f m4bf filename.mp3

  • Nice. After reading the man page, I see that it can be used to convert image file formats:

        sips -s format png photo.HEIC --out photo.png
    

    or resizing:

        sips -z 300 600 original.jpg --out new.jpg

  • sips looks really cool, thanks for pointing this out.

    Not gonna lie, I missed this because I thought it was related to macOS SIP, System Integrity Protection. Which is something I am deeply uninterested in.

I've never heard of networkQuality, that's seems like quite a useful tool.

  • Unfortunately, it's not actually very useful, as whatever server they're using on the other end is severely constrained. networkQuality gives me:

    Downlink: 884.856 Mbps, 198 RPM - Uplink: 13.238 Mbps, 198 RPM

    whereas speedtest (whether to the official speedtest server OR a friend's home server in their basement!) gives ~700 Mbps uplink.

Here's a handy use I've found for mdfind.

Say you've got a directory that has scripts or data files related to some thing you do. For example I've got several scripts that I use when I scan books with my book scanner. I only need these when doing book scanning stuff so don't want to put them somewhere in $PATH. I want to be able to easily run them from scripts that aren't in that directory, but I don't want to hard code the path to that directory.

Solution: in the directory with the book scanning scripts I make a file named ID that contains a unique string. I currently use 16 byte random hex strings [1].

I have this script, named find-dir-by-ID, somewhere in $PATH:

  #!/bin/zsh
  ID=${1:?Must specific ID}
  IDSHA=`echo $ID | shasum | cut -d ' ' -f 1`
  mdfind $ID 2>/dev/null | grep /ID | while read F; do
      FSHA=`shasum $F | cut -d ' ' -f 1`
      if [ $IDSHA = $FSHA ]; then
          dirname $F
          exit 0
      fi
  done
  exit 1

If some script wants to use scripts from my book scanning script directory, it can do this:

  SCRIPT_DIR=`find-dir-by-ID 54f757919a5ede5961291bec27b15827`
  if [ ! -d $SCRIPT_DIR ]; then
    >&2 echo Cannot find book scanning scripts
    exit 1
  fi

and then SCRIPT_DIR has the full path to the scanning script directory.

The IDs do not have to be hex strings. If I'd thought about it more I probably would have made IDs look like this "book-scanning:54f757919a5ede59" or "arduino-tools:3b6b4f47bf803663".

[1] here's a script for that:

  #!/bin/sh
  N=${1:-8} # number of bytes
  xxd -g $N -c $N -p -l $N < /dev/urandom

  • Why not just a directory with subdirectories by ID? No mdfind needed, no problems with just-created directories, no wait, etc

    • You mean something like having ~/well-known-stuff and under that having a 54f757919a5ede5961291bec27b15827 directory with the book scanning scripts and so on?

      That could work fine, but generally the directories I've used this on are directories that I want to have somewhere else, and with a reasonable name. Usually the directories came first and various other things in fixed relative positions were using them, and then later I wanted to use them from elsewhere and added the ID.

      I suppose ~/well-known/stuff/54f757919a5ede5961291bec27b15827 could by a symbolic link to the original.

      The mdfind approach does have the advantage that if I reorganize things and move the directory it keeps working.

    • Beyond opaqueness, which could be overcome by simply prefixing the ID with something descriptive, I guess it helps avoid creating excessively long paths that might create problem onncertain filesystems, which is rarely a problem these days, but I have been bitten by it a half dozen times over the last few years. In a couple instances causing (recoverable) data loss.

There is also pmset which is very useful (since macOS doesn't give a UI counterpart) https://support.apple.com/en-am/guide/mac-help/mchl40376151/...

  • I have this .zshrc function to track the battery and charging, which uses pmset:

        function batt-info() {
            echo
            system_profiler SPPowerDataType | grep Wattage | cut -c 7-
            echo
            pmset -g batt
        }

  • I've found reliably "turning on" with pmset to be hit or miss. I can't remember the gotcha I ran into if it was that you had to have your laptop lid open or something else...

    • Yes. If macOS is asleep or idle, pmset won't get triggered. It is best used with caffeinate. I used it for a museum installation where every night I'd turn it off before they cut the electricity and turn it on in the morning just after electricity is switched back on: it was a way for me to be in peace with my conscience (though after some tests, the mac didn't bother getting switched off by cutting out the electricity).

My non-built-in CLI utility recommendations, none of which are macOS specific:

* atuin - TUI for shell history, backed by SQLite - https://github.com/atuinsh/atuin

* LSD (LSDeluxe) - rewrite of `ls` - https://github.com/lsd-rs/lsd

* ripgrep - https://github.com/BurntSushi/ripgrep

* fzf - command-line fuzzy finder that enhances file search, command history search, and more - https://github.com/junegunn/fzf

Notes:

- To get pretty extra file/folder symbols with LSD, you'll probably need to install some special fonts.

- You can use `fzf` and `ripgrep` together.

pbcopy and pbpaste are handy, for a version that works over ssh connections there is osc: https://github.com/theimpostor/osc

  • since I switch between linux and macos a lot I wrote a dotfile function called "clip" that will work the same on both. nice thing is it will automatically paste if nothing is piped to it to copy so there's no need to use separate commands... although I just realized it might be nice to have a "passthrough" mode that both copies and pastes if you add this to a pipeline in order to capture some intermediate part to the clipboard

        if [[ "$(uname)" == "Darwin" ]]; then
          clip() {
            [ -t 0 ] && pbpaste || pbcopy
          }
        else # assume linux if not macos
          clip() {
            [ -t 0 ] && xclip -o -selection clipboard || xclip -selection clipboard
          }
        fi

    • That's handy, thanks! `osc copy` may also take args for files to copy to the clipboard, but in the absence of that and no data on stdin it maybe should switch to paste.

    • I went the route of managing a different set of dotfiles for linux and macOS. Same repository, just different branches.

      Also, falling back to using oh-my-zsh functionality.

  • At my job I have to work with a lot of JSON that's usually minimized. This command has single-handedly saved my sanity:

    $ pbpaste | jq | pbcopy

    Then I can paste it into whatever text editor I want and it's all nice & pretty-printed for me.

    Bonus is that I don't have to change the command at all, just copy the minimized JSON to the clipboard (say from DBeaver, for example), then hit the 'up' arrow and enter.

    • I never knew that jq without any arguments pretty-printed JSON. Very useful, and great tip to combine with pbcopy/pbpaste.

And for those of us who have to support macOS users:

   sw_vers (to display the version/build of macOS),

   dsenableroot (to enable the root user), and

   say "phrase" (useful for freaking out users over a remote SSH connection)

I got this trick from someone on the Internet:

$> long_running_command && say "Witness me, for I am done"

  • Yep, I do this in a tmux session. But sometimes I'm on a call and forget and the ROBOT VOICE says "DATABASE DUMP IS COMPLETE" and I jump out of my skin.

A fun easter egg in the "say" command is that "OS X" is said as "Oh Ess Ten".

You can also change voices with -v. My favorite is "cellos" since it sings to you.

  • It actually understands roman numerals, to a certain extent. E.g. `say LVIII` will say "58". However, `say MCMLXXIX` speaks some gibberish that ends in the word "six", for some reason.

    It also knows how to say numbers up into the trillions but not more than that (although I feel like it used to).

  • Other cute `say` tricks: muck around with OSX Default Speech Voice (to a siri-ish voice) and you can invoke that from the CLI.

    I hacked together a little script for demo recording like:

        START_FROM="$1"
        STARTED=0
        function transcript() {
           ID="$1"
           TO_SAY="$2"
           if [[ ...STARTED || START_FROM && ID... ]]; then
              STARTED=1
              say "$TO_SAY"
           fi
        }
    
        transcript "STEP001" "This is a test"
        transcript "STEP001b" "of the emergency broadcast system"
        transcript "STEP002" "This is only a test, if this was ..."
        transcript "STEP003" "...etc..."
    

    ...and then I have a hardcoded `--output` which will then change the invocation to `say -o "$ID.wav" "$TO_SAY"` and output audio files.

    That way I can iterate on voiceover scripts, eg: `./demo-script "STEP002"` => `./demo-script --output`

    It's really helpful for iterating on pronunciations, eg: `transcript "STEP005" "In case of Four Hundred and Four errors..."`, I can just "skip to" and iterate against that line (and subsequent ones), or "skip back" and hear it in more of a flowing context.

    ...even if I don't end up using the `say`-generated audio, having a transcript (and even pacing) that I can just read through with my own voice is super helpful.

Want to scan the local wifi networks from the command line, and get useful information like signal strength?

/System/Library/PrivateFrameworks/Apple80211.framework/Versions/A/Resources/airport -s

I set a shell alias so I can just do `airport -s`. I've no idea why this is hidden away inside some framework and not in a directory which is in the normal path, but there you go.

  • FWIW that appears to be soon deprecated according to MacOS 15.2:

    WARNING: The airport command line tool is deprecated and will be removed in a future release. For diagnosing Wi-Fi related issues, use the Wireless Diagnostics app or wdutil command line tool.

    • Oh, that's a pity. I'm pretty bad at keeping up to date on MacOS releases, but I should probably start figuring out `wdutil` so that my muscle memory is adapted before I've got no choice!

      1 reply →

What I'm really missing still is a cli to iCloud stored passwords. AFAIK 'security' cli can't access the credentials stored in the cloud. This would be helpful to store secrets outside of git but would still allow scriptable access to them similarly as 1password cli 'op' has.

Looks like a lot of these have linux equivalents that could be aliased. I wonder if anyone's made a set of those for regular macos users who occasionally use something else.

> If you store your secrets in the Keychain (and you should!)

As part of the OS, Keychain suffers from the same sorts of sharp edges as using a built-in interpreter. An alternative is to use a password manager. Below is an example of the tools available in one.

https://developer.1password.com/docs/cli/get-started/#step-1...

  • Would you mind expanding on what these "sharp edges" are that you're warning about? I'm not really sure what you mean by this. Keychain has served me (and I imagine many others) really well for a while.

    locate 

https://ss64.com/mac/locate.html

locate searches a database for all pathnames which match the specified pattern. The database is recomputed periodically, (about once a week) and contains the path-names of all files which are publicly accessible.

  • The database is not set up by default.

    Your better bet on MacOS is to use `mdfind -name ....`

    • I have that aliased to `locate` on mine. It's not exactly the same but it gets me 99% of the way there.

I just remembered at a previous job where we were always paired and used team rather than individual iMacs for development that as a prank I ssh’d into the other iMac from my team and periodically would run open on a random URL which caused a Safari window to mysteriously pop up their system.

pbcopy and pbpaste are among my top commandline utilities. If you are running linux in WSL, you can add the following to you path to recreate their functionality:

    # pbpaste
    #!/bin/bash
    # Adapted from https://www.techtronic.us/pbcopy-pbpaste-for-wsl/
    powershell.exe Get-Clipboard | sed 's/\r$//' | sed -z '$ s/\n$//'
    exit 0

and

    # pbcopy
    #!/bin/bash
    #Adapted from https://www.techtronic.us/pbcopy-pbpaste-for-wsl/
    <&0 clip.exe
    exit 0

I'm sure everyone knows this, but `open` has an equivalent on Linux: `xdg-open`.

  • Except xdg mime types are easily heist-able by programs and there is no universal truth for it.

    Install vscode on Linux and have fun with it hijacking opening directories. Which is easy enough to fix (with a bit of know how) or work around but annoying.

    MacOS is far from perfect but open always doing what I expect is a good example of what Apple did well.

TIL: caffeinate

Very useful.

afconvert is pretty nifty for audio format conversion.

pbcopy is my favorite. Almost enough to prefer a mac over my usual linux stations, but you can get that on linux easy enough.