Articles like this are fun but they all come from posix shell syntax being fundamentally bad for scripting/programming. All the piping stuff is great, of course. And the overall ecosystem is great. But the interpretation of the script itself working by a series of string substitutions is a mechanism we wouldn't accept in a regular programming language. And there's no excuse for it really, except that shell syntax is really, really old.
For example, what does `$foo` mean in shell syntax? In any reasonable language (perl or powershell, for example, or python if you drop the `$`), it's an expression that evaluates to whatever value's inside that variable. In shell, `$foo` isn't an expression in that sense, and what it does depends on what's inside it via a variety of string substitution rules.
This is the main reason we have arcane articles like this.
Old doesn't imply arcane. It instead can (and does, in this case) mean that it won out over the course of decades against other less worthy alternatives.
Although you can write "programs" in sh, the shell syntax was never meant to be anything like a systems or application programming language. It was meant to efficiently automate systems tasks. It is kind of like the fork lift of computing. It does it's job very well, is bad at doing anything else, yet no other kind of vehicle can do it's job nearly as efficiently.
Comparing the shell to C, Go, Rust, Python, or JavaScript is crazypants. It has a different job than those, so of course it will look and feel different!
There are plenty of alternative shells that don't suffer these problems. Posix syntax didn't win because it was better - it's the classic 'worse is better'.
> Old doesn't imply arcane.
Indeed so. For example, awk is about the same age but doesn't suffer the same problems.
> the shell syntax was never meant to be anything like a systems or application programming language
A programmable shell is not only like a programming language, it is a programming language. Systems tasks have some specialised requirements but, overall, scripting is programming. There is nothing in the problem domain that requires the posix string-substitution problems.
If fish or oil had existed back then, they would likely have won, since they are better.
I think Bash won as Bityard says above, in the sense that COBOL won, which was because it was everywhere (approximately) and then when another system was being built to do something in COBOL’s niche COBOL was the obvious choice. Once it was sufficiently established it won by inertia and absolutely not because it was the better language. Or maybe it was the better language in its time, but it certainly is not now. And note I’m only talking about COBOL’s niche, of mainframe-based processing in the banking, insurance, state and federal payroll and similar areas.
> It was meant to efficiently automate systems tasks.
Shell was designed to launch processes easily. Unix philosophy pushed to make every function an application. And that's ultimately what shell "syntax" is.
There's shockingly little syntax in the shell. A lot of what we think of as "shell syntax" is actually full blown (standard) programs which the shell is launching. My favorite of which is `[`.
What makes shell unique vs all other programming languages is it puts launching processes first. It has no friction there because if it did, it wouldn't work.
This is also what I think is gross about shell. Every other scripting language has at least some friction when launching other processes and that's usually a good thing IMO.
One of these are not like the other. Python is perfectly serviceable as a shell, to the extent that one could largely replace bash with the Python REPL if they wanted.
I disagree. I see shell syntax as an “inversion” of normal quoting rules. Normally quoting is the default but to achieve anything special you have to use an escape. Conversely, in shell, quoting is not necessary but any special character will likely have unwanted behaviour unless you quote.
This is handy for quick scripting as a natural extension of a command line interface. A counterexample might be an equivalent script using the Python subprocess module. For a simple script, the required extra quoting overhead is not worth the trouble. For something complicated it’s essential and a shell script is no longer suitable. For something in the middle, a carefully crafted shell script (eg. one that passes shellcheck) may be the most suitable solution depending on the situation.
Both cases have their place. It’s about using the right tool for the job.
> fundamentally bad for scripting/programming […] a mechanism we wouldn’t accept in a regular programming language
I don’t agree with this assumption that shell should use mechanisms we accept in “regular” programming languages, whatever that means. And why not the other way around? Why do we accept mechanisms in programming that we wouldn’t accept in a shell? There are layers of assumption in your assumption.
I also disagree with lumping scripting and programming together. Those are two different activities, and shell is better at scripting than programming, and it’s better at scripting than programming languages.
Try using python as your shell, and you’ll find out that python is fundamentally bad as a shell REPL. The shell is necessarily different from python or C++ or <your_favorite_programming_language>, starting with the ability to run command lines by typing them, with zero extra syntax. That is something regular programming languages can’t do, and it leads directly to the need for string substitution rules.
If you want a shell made out of a good programming language… make one! People have been thinking about this for decades and nobody has come up with a good one, because shell languages like bash are better at shelling than programming languages.
Yes a large percentage of bash scripts fails if you feed them filenames with spaces or other special characters. Or a large amount of filenames near the Unix maximum commandline length.
Bash et al are great for command line use, but produce a situation of really bad engineering hygiene when used in scripts.
Do not use. Stay away. Bash et al considered harmful. Red flags on job interview when referenced.
Whilst I agree that Bash makes it far too easy to do it wrong, it's not too tricky to write scripts that don't have those issues.
Possibly the most important tip is to use ShellCheck (https://www.shellcheck.net/) as a linter on your scripts and work through the various warnings to eliminate them. For the rare times that ShellCheck is overly cautious, you can remove a specific warning with a comment before the offending line such as
# shellcheck disable=SC1091
Also, take time to look through Greg's Wiki as it is possibly the best Bash resource that discusses the various pitfalls: https://mywiki.wooledge.org/BashPitfalls
As a Bash script writer, I disagree with your "red flag" interpretation as that'll mean that you'll end up with poorly written Bash scripts along with a hotch-potch of different tools to achieve the functionality (e.g. different versions of Python required for older scripts and newer ones). Knowing Bash is also very useful for writing Dockerfiles.
> If the pipeline does not begin with the "!" reserved word, the exit status shall be the exit status of the last command specified in the pipeline. Otherwise, the exit status shall be the logical NOT of the exit status of the last command
Do you have a real-world example where we would like to preserve `$?` after the `if` (which `if` specifically branches on the result code of the command it tests)? Where we want to use the actual return code that the `if` did not handle itself.
I am asking because I have never needed preserving the `$?` outside of an `if`, nor have I ever seen this in somebody else's scripts, and I'm curious.
I once created a set of shell functions which I wanted to have a docstring-like functionality. The solution I came up with was to have each shell function start with the : command with a string as an argument. Since : is an actual command, not a comment, it was preserved as part of the function, and could be extracted at runtime using relatively simple parsing to do introspection.
Example:
foo(){
: "This is a docstring for the foo() function"
bar --verbose | baz --quiet
}
I am not a huge fan of most of these, but a few do seem useful.
: "${1:?missing argument, aborting!}"
I wouldn't use this because I would want to give $1 a name for the rest of the script, so I would assign. But it can be a nice way to give a clear error for missing required environment variables.
Many of the others (like truncating files) are probably more clearly written with dedicated commands, but may come in useful if you are going to extreme lengths to avoid dependencies outside of the shell.
You are very much correct and I 100% agree with you, I have updated the first example to include a snippet where a proper env-var is used to show of the automatic diagnostic.
For a short-lived script, the `${1:?missing argument}` stuff may be useful, but usually, I want to print some longer usage or help text in the error case.
$1 might also just be the first parameter for a function call, so you might not need that parameter to have a specific/readable name if the function is not very complex or long. I have plenty in my dot-files that are just one-line functions that don't need to be three-line functions just to give the single parameter it accepts a name.
"Shortcuts" like :? that tie together two unrelated things (checking a variable for a condition, expanding the value of that variable) drive me up the wall. It makes expressing just one of the two things harder than expressing both, leading to contortions when you want just one, like this use of :.
I use this for setting overridable defaults (this should hopefully actually be in the article I haven't read yet)
: ${DEBUG:=false}
debug is not only false, but garanteed to be set to something so that elsewhere the places that use it don't need extra syntax and checking in case it's empty, and yet you can just set it from the parent environment to override without touching the script or adding a commandline args parser.
And do me, reading this is almost as easy on the brain as just plain DEBUG=true.
Even if you aren't familiar with the extra syntax, like you're someone else in the future who needs to look at it, the two important words pop out, and probably don't mean NOT because whatever those extra bits mean, there's no !. So you get the idea well enough & cruise on.
...[edit] yep it's in the article.
but one thing isn't, exactly
In other words, maybe you should have figured out some other way to write the whole construct, but IF you want to write the if/else this way maybe for just readability or organization, and you want the else-block to only hinge on the initial condition and not also on whatever the then-block might do, then you need a safety-true at the end of the then-block.
That seems unsafe. It assumes that the editor is started via a shell command, and not executed directly. But there is no actual binary /usr/bin/: but there is a /usr/bin/true; I would use that instead.
It assumes git works the way that git works, and that git will not break backwards compatibility in the future. I think the same can be said of any alias in git.
I actually did absolutely need it recently when I golfed together a shell script that is simultaneously a valid YAML file [0]. Sometimes having no-op tokens is nice!
Why take a perfectly readable if-statement and turn it into something, 99.9% of people would need to lookup. Concise != better. You can make it one line with:
the people writing it are doing it for themselves probably. i wouldn’t expect it to survive a code review. the necessity of using esoteric bash features or syntax is a pretty good smell that you should be using something else imo.
I believe you missed a semicolon after `exit 1` and before `}`. `}` closes the opening `{` only at the start of a command (POSIX rules; don't remember now if bash recognizes it when it's just a command argument).
Off-topic, but I am reminded of Larry's First and Second Laws of Language Redesign (which Larry Wall discovered/stated when he designed Perl 6, which is now Raku):
1. Everyone wants the colon.
2. Larry gets the colon.
This "hidden knowledge" is fun to read but pain to remember and use, especially when working with multi-platform environments //
Switched from bash to plain python scripts for shell stuff everywhere several years ago, and never looked back into bash zoo anymore. Stable syntax across Win/Mac/Linux, no bash/zsh/msys2 obscure differences, normal errors and Clause writes scaffolds quick and flawless anyway
Calling all the command line tools is cumbersome, the Python cloud APIs are verbose compared to the cli tools and often have very different concepts for how they interact with services.
But I agree that there is a need for a better bash and zsh is nice but not the upgrade that is needed
The extent to which the script author gets to feel smart and efficient is exactly the extent to which the future reader of the script gets to feel like an idiot.
So true. I think most of us who learned "advanced" bash is after being tired of feeling like an idiot looking at long bash scripts.
Usually, those scripts are written by people who didn't learn the "weird tricks", and use global variables (they didn't learn that `local` creates local variables), for example.
In bash, many times the "dumb way" takes 10 times more lines, and it's probably buggier than the "smart tricky way". So, in the end it's your choice what side you pick, and when to stop. Or, use a "real programming language" (I hate when people say this LOL)
After a while, you get a bit of stockholm syndrome, and you're just fine with bash as an orchestration language, and lean into unix principles. I accept its limitations, and I think it keeps me on my toes as it repels bloat.
idk if you were expecting an answer, but bash lives in this liminal space in the tower of languages that is worth a thought.
The only upside of weird bash tricks is there's so much bash out there that the LLMs are really well trained on it - so you won't have to read the script, just the tests.
life is way too short to deal with this nightmare of a language and its 50000 footguns for anything longer than a 2 line script, especially in the age of LLMs. Just write a python/TS/any real language script instead. Bash is great for the command line, it should be limited to use there.
It turns out that LLMs are really good at writing bash too. even perl! maybe we should rethink some of these lost bits because we no longer need to worry about the arcane parts.
Define "really good?" I don't think I've ever had them produce a script that I didn't need to correct in some way. They can write it, that's true, but we still need to be able to read it.
Bash scripts requires "set -eu" at minimum, when written by LLM or human.
It's the only language terrible enough to make the default behavior ignore undefined variables, commands, and execution errors, and happily continue executing whatever was produced by me smashing my hands on the keyboard, until the end of the file, while returning an exit code of 0, claiming complete success.
I have seen them write massively more complex bash scripts than any normal person would ever attempt, but the failure rate is absurdly high compared to when they write sane typed languages.
I have some absolutely amazing bash scripts that I would never have contemplated making myself in bash. And they'd only have been somewhat better in python. Claude for the win!
I find LLM's too fall into the trap of writing a bash script for a task that clearly needs to be implemented in an Actual Language with Real Data Structures. For example, ask an LLM to bring up a SQL server with some schema + data preload step, and it will write a profoundly long bash script to do that task, every time.
my experience as well. claude produces functional but over-engineered scripts fairly frequently with often very questionably useful safety checks, especially for powershell.
a common tell of ai generated powershell is a script that has dedicated functions to check types, often via several methods, and happily prints the output of the checks to shell. i do not get why it does this but it often adds dozens of lines that really serve no purpose but to make the shell output look fancy
Try being on linux and using zsh, it constantly fucks up scripts. Having it run commands via SSH into a windows machine is even more a comedy of errors.
I have to put in every claude.md that the shell is zsh. It's reached the point of annoyance I might just go back to bash.
TS compiles to JS, so when you actually execute a script, you’re just starting Node, Deno, etc. with a JS file. Any of those are probably faster to start than Python. Deno in particular starts in around 15-60 milliseconds. Even if it has to compile the Typescript - which is a one-time operation if the script isn’t changing - it still typically starts in under 250ms.
What other language is it as easy, fast, and painless to work with file contents? I can toss data down and lift it back up with some <> symbols and a path, none of Python's "errrm, how would you like to open that file " handholding. And why the hell would I want to go learn TypeScript!?!?
Sometimes I feel like the people on HN repeating the trope "uhhh if you need ____ go learn a real language!" are just bad at writing shell scripts.
I have probably the worst use case, but I like it. I have a very specifically structured ZDOTDIR, and I write everything in a way that is self-documenting.
After a while, you probably know more commands and utilities than you know what to do with, and you'll forget they exist when you need them. In order to not waste time looking for a program for a particular and infrequent purpose, I create "do nothing" aliases like `: alias f3probe` so I can realize I just forgot I already have something for it. Nice predictable pattern to grep with `^: alias` to look through all of these.
$ ./script.ps1 Dave
Hello, Dave!
$ ./script.ps1 -name Dave
Hello, Dave!
$ ./script.ps1
cmdlet script.ps1 at command pipeline position 1
Supply values for the following parameters:
name: <cursor here>
Or non interactively:
$ pwsh -nonint ./script.ps1
script.ps1: Cannot process command because of one or more missing mandatory parameters: name.
PowerShell needs to become more ubiquitous. There are so many things about it that once considered in greater detail make so much sense. Parameter setup, typing, and naming. Flow control. Object-oriented scripting. Built-in parsing for recursive data types like JSON, HTML, XML.
And in my opinion, the most slept-on: the fact it runs on the CLR and direct access to .NET objects and types which means access to P/Invoke and thence the Windows API. One can write business logic in the fast language and write a nice CLI wrapper around that in the natural shell language, and not worry about painful FFI unlike everyone else trying to fit Python or Bash into whatever world they're using.
The typical counter to this will be: PowerShell is verbose, PowerShell used `curl` as an alias to Invoke-WebRequest instead of the Real Thing™. Neither are real arguments.
I'm curious, have any good examples of projects in the ~1K LOC range of PowerShell, so I could get a taste for how something like that would look like? Beyond that I tend to go for "real" programming languages, or just start with Babashka, but still curious to see projects other PowerShell fans would consider "well written/designed" :)
I wrote my first real `.ps1` the other day for auto-installing all dependencies needed to run a `gitea`-runner on windows for windows builds; powershell - felt like the lover I never had. And the documentation in readable comments at the top that just.. generates usable docs? Damn, damn, damn.
There is a part of me that low-key wanna try that as my daily driver for a week or two. But with that said, I'm a zsh vi-mode guy - always have been, always will be.. but I'd happily take powershell on a romantic getaway every once in a while!
PowerShell is a bad choice for the system shell because it's too complex. I'm not trying to justify all the quirks of Unix Shell, but minimalism is a very important feature of such a tool.
Another important feature of a tool like this is the ability to tolerate errors: I can't imagine a Linux today that would be able to even boot if the shell was extra pedantic about errors. A lot of mostly irrelevant things routinely fail on boot and during normal operation. Stamping them all out is an arduous... well, basically, an impossible task for practical purposes where releases are expected to come on time, where users may manipulate configuration in gazzilions of unpredictable ways.
PowerShell is just another language in the same box with Python, Perl, Ruby and many like that. It's not a good language, if you decided to reach for that box. Probably not the worst either.
System shell, however, isn't meant for writing entire applications. Writing applications with elaborate command-line interface should be left to languages that can properly address this problem. PowerShell is trying to be there, but it doesn't hold a candle to its "older brothers" who can, indeed, design a very robust command-line interface, often using a dedicated library for it.
PowerShell appeals to the novice crowd who are very enthusiastic about automatic checks in their code: the benefits are on the surface, the downsides are difficult to assess. This is in line with other Microsoft software products / languages which target novice programmers by implementing as many as possible of the highly-advertised features without regard to the overall usefulness of the product (think about C# or MS Office suit etc.)
> System shell, however, isn't meant for writing entire applications.
Programmable shells are, in fact, for programming. That posix shell syntax makes it impractical for meaningfully large scripts is something you just accept. And, realistically, it is something you have to accept.
Minimalism isn't why posix shell syntax is bad. For example, awk and jq are minimal but don't make the same mistakes.
Powershell was never designed to be your system shell (Windows doesn't really rely on one) but the main reason it can't be the shell is startup time, which rules it out of wrapper scripts. Its having lots of features might contribute to that but it's not the problem per se (being built on .net is the reason - that and startup time not really mattering on windows). Its lacking the syntax problems of posix shells is very much not a problem. Powershell's error handling is just fine, by the way.
The startup time of even pwsh is too slow to be usable in many scenarios.
PowerShell's not going to be the system shell on your *nix box but it wasn't designed to be. But your posix shell isn't the system shell because it's better. It's pretty much the ultimate expression of worse is better.
That's why there's an industry of alternatives without the faults, such as oil, elvish, fish, nushell. And why awk is so popular.
> Another important feature of a tool like this is the ability to tolerate errors: I can't imagine a Linux today that would be able to even boot if the shell was extra pedantic about errors. A lot of mostly irrelevant things routinely fail on boot and during normal operation. Stamping them all out is an arduous...
You seem to be implying PowerShell aborts on any error. That's not the case:
My main issue with attempting to replace Bash with other shells is that they're often not installed by default on different Linux flavours and they don't/won't have the same longevity as Bash. Try running PowerShell on a 20 year old Linux system or maybe find out if there's some version issue with running an old script 20 years in the future.
- it creates the file if it does not exist, not merely truncate. as a tutorial kind of blog post this incomplete description matters IMO.
- it would work the same without the colon (similar for default variable assignment examples). we generally strive not to have "extra" things, like useless use of cat.
- educationally it's useful to demonstrate that redirection, like parameter expansion, works before the command executes (the null command in this case), but the article doesn't explain that at all!
otherwise i <3 this article. some uses of colon i had never thought of or seen before. like file truncation, not sure i'd use them but it was cool to see them.
Whats the advantage of the colon for the truncation example
>file1 >file2
versus
: >file1 >file2
I've done quick and dirty, interactive truncation like the former for many years, no colon. But I would not use it in scripts
According to https://www.in-ulm.de/~mascheck/bourne/ SVR4 (1989) had a bug when using this method in a for or while loop and this bug showed up in a SunOS 5 variant, too
Apparently, early in the shell's evolution, : was used as a comment marker before # was added
Single quotes could be used to prevent undesired behaviour
I always wanted to learn more about scripting. But today I am not as passionate as before because LLMs write working scripts most of the time. I am wondering if it is true for most programming techniques and quirks. Are we going to write code to solve low level problems?
The other day there was a blog post about learning SIMD. I think in future "programmers" will just nag about the speed of the program and the coding assistant will eventually introduce SIMD to the source code.
It is a little sad but we have to go with the current if we want to survive.
I still think that learning the shell environment makes a lot of sense these days. But mainly for interactive use. It's amazing the scripts your LLM can make you with, say, fd and fzf.
I can see crafting line by line when I’m retired or for a personal project. It’s eventually going to be like the people now who still operate restored manual linotype machines. “Look here I have the last functioning one in the midwest. See how the plates work…”
As a go-to idiom for a writability test, it gives me pause. If the file didn't exist, we created a zero-length one. That might be okay if we are going to write to it anyway as the next action.
If we are testing because we intend to overwrite it, why not just "> result.json" (which is by itself an idiom for truncating a file to zero length).
When would we every do this? Maybe before some command which takes the file name as a destination file argument rather than using output redirection, and which performs a lengthy computation before trying to open the file for writing. We can catch the permission error early.
I don't think I've ever coded such a test; normally you just do the operation that writes to the file and let that fail.
In POSIX C, there is a function access() for doing these kinds of tests. But it has a special purpose: it is meant to be used by a setuid root process to perform a permission test as if it were the real user/group (the one which elevated privilege to root). I.e. it's not can we do this operation, but should we do this operation (would we still be allowed, if we dropped privileges back to the original user).
They are NOT superflous, and all you need to prove it is `zsh` (but there are others that follow suit in similar fashions):
zsh% echo "hello world" > data
zsh% < data && echo "READABLE" # <- will print the contents of data
hello world
READABLE
zsh% : < data && echo "READABLE" # <- this will not
READABLE
So, if you want something that "everyone can use" without going into details about the difference between commonly used shells.. you'd use the null-command.
---
and given that we use the null-command, it _WILL_ behave different with or without subshell.. and all you need is `bash --posix` to prove it:
% cat subshell.sh
#!/bin/bash --posix
( : < missing.json ); echo AFTER # <- will echo AFTER
% ./subshell.sh
./subshell.sh: line 2: missing.json: No such file or directory
AFTER
% cat no-subshell.sh
#!/bin/bash --posix
: < missing.json; echo AFTER # <- this will not
% ./no-subshell.sh
./no-subshell.sh: line 2: missing.json: No such file or directory
% : ^- apparently.. there is a difference
The output above is not truncated, `no-subshell.sh` will stop executing due to the broken read.
---
One should never trust things just because they are written, but that also applies to comments on HN. Originally when I read your message I actually thought I made a mistake, I was very close to writing an apology comment and adding a note to the blog post, but not close enough - I had to test it again.
I'm thankful for the watchful eyes and scrutiny when reading things online, that's good - keep it up, but your message is factually wrong - on so many levels.
This is required by POSIX in the section "Consequences of Shell Errors". If a redirection error occurs in a special built-in command, a non-interactive shell is required to exit.
The bare redirection without : does not have this problem:
< missing.json; echo AFTER
Therefore, in a POSIX-conforming shell, we do not need to wrap it in a subshell.
> but your message is factually wrong - on so many levels.
How many? Can you count the levels? I didn't know that in zsh "< file" dumps to standard output, which is suppressed by :. If I used Zsh, I would know that sort of thing.
It’s very common that the most upvoted comment on a post is a mistaken correction. People seem to love comments seemingly proving that the post is wrong without actually checking anything. And once it has enough upvotes a big discussion may follow up with personal attacks on the author or other questionable comments, while the people trying to point out that the post actually is accurate get either buried under the critic storm or downvoted to oblivion because people just don’t like the truth. This happens in every controversial topic.
Maybe zsh still reads the entire file and copies it to /dev/null, which would be a severe performance problem for large files.
The colon by itself, without the subprocess, also prevents the dumping to stdout:
zsh% : < data && echo READABLE
However, with the subprocess parentheses, we can redirect the nonexistence diagnostic to /dev/null, which I don't see mentioned or exemplified in the article:
zsh% : < nonexistent 2> /dev/null && echo READABLE
zsh: no such file or directory: nonexistent
zsh% (: < nonexistent) 2> /dev/null && echo READABLE
The normal way of testing whether something exists and is readable that you would actually use in production script looks more like this:
I would say that "if you want something everyone can use", the -r test would be the first candidate.
BTW, why not bring up the strawman of tcsh?
tcsh% < nonexistent && echo READABLE
Invalid null command.
tcsh% : < nonexistent && echo READABLE
nonexistent: No such file or directory.
Yes, if we want an obfuscated readability test which works literally in any shell that is currently still in deployment in systems that permit new scripts to be installed and run, it looks like do need that colon.
However, fish doesn't like &&:
fish> < nonexistent && echo yes
fish: Expected a command, but instead found a redirection
fish> : < nonexistent && echo yes
fish: Unsupported use of '&&'. In fish, please use 'COMMAND; and COMMAND'.
: < nonexistent && echo yes
^
Does it support test -r?
fish> test -r nonexistent
fish> test -r nonexistent; and echo yes
fish> test -r /etc/hosts; and echo yes
yes
Not parentheses though:
fish> ( test -r /etc/hosts ); and echo yes
fish: Illegal command name “( test -r /etc/hosts )”
We clearly have to restrict the idea of what "everyone can use" to POSIX-like shells; there is no getting around shell differences absolutely, other than for perhaps trivial command invocations.
I've read through all of the examples in the article & they all seem to serve to sole purpose of turning readable multiple line code into one-liners.
One-liners are a cool little artifact of early shell culture & are sometimes still useful today if they're short to avoid the readability problems of `/` when copy pasting a quick shell command to run, but they have no place in scripts.
None of this seems useful to me.
> if you are like me and prefer less typing (gotta go fast)
Bash scripts should only be used for quick and dirty tasks where brevity is a major benefit. Don’t pretend bash can be readable and maintainable. If you want that use another language that sacrifices brevity for clarity.
First I agree that bash scripts should only be used for quick or small (& ultimately mostly personal unshared) tasks. There's absolutely no need for it to be either "dirty", & as mentioned in my post, brevity has no material benefit in this context.
I want my personal local utility/productivity scripts to be readable: quick to write & quick to modify on the fly. Brevity doesn't help here - wpm optimises for natural language typing & that translates better to idiomatic logical block structures than to symbol-heavy one-liners.
I also want the same for the small bash snippets in my CI jobs - this is a particular example where brevity is actively bad: this encourages folk to inline their bash snippets in yaml (no syntax highlighting & unlintable) when they should be packaged in script files in CI directories.
there are some early unix tapes floating around, and in those early shells, i'm fairly certain the colon was one of only two special-cased code paths after the command line was parsed. does anyone recall more specifically?
DESCRIPTION
Goto is allowed only when the Shell is taking commands from
a file. The file is searched from the beginning for a line
beginning with `:' followed by one or more spaces followed
by the label. If such a line is found, the goto command
returns. Since the read pointer in the command file points
to the line after the label, the effect is to cause the
Shell to transfer to the labelled line.
In the Bash shell, true is a builtin. It may be a builtin for other POSIX-compliant shells. false is also a builtin.
The key difference is that : is required to be builtin. There are several good reasons for this. For example, syntax: the shell can single out this special character syntactically before searching $PATH.
Also, ":" is a disallowed or problematic character for certain filesystem types. If your shell attempted to omit this builtin, it could not always rely on an external command file by this name.
Lastly, people keep bringing up fish, but it is not a POSIX shell, so its similarities in syntax and operation are coincidental.
I really hate bash because of its unreadable syntax, and this does not help it in any way.
We have some large bash scripts in my company, ~10,000 LOC spread across multiple files, all sourcing each other and what not. It is truly hard to read bash, which means it is truly hard to maintain bash, which means that when the one person knowing the bash scripts in your company goes away, you're in for some "fun".
My point is, these quirks are not useful, except for some bash enthusiasts.
Its not "unreadable" syntax, it's terse and unfamiliar. There are many languages with terse and unfamiliar syntax to me that I struggle to read, but only because I'm unfamiliar. In fact, most programming languages have unreadable syntax at first, save for maybe...AppleScript? Maybe Scratch. Even Python's famous readability isn't guaranteed, some of its functional programming facilities are just as arcane and unreadable as *sh.
I spend about 50% of time working time in the shell. I didn't find anything in TFA unreadable.
I usually skip the get option builtin, and use : as...
while : ; do
case "$1" in
"") break;;
-f|-foo) shift; whatever;;
*) usage; exit 1;;
esac
done
For this... instead
if something; then
true
else
echo ERROR
exit 1
fi
Using : would be too much here.
For anything else including json etc. I usually go to duckdb. Awesome support, single file install, readable, easy to maintain.
Powershell on Linux or Unix? Just another huge dependency if you manage 1000s of machines, and good luck finding a Linux gal/guy wanting or able to touch pwsh without chemical grade gloves.
Yeah, I don't see pwsh finding a place in the ecosystem. Even oil or fish struggle. Their place was mostly taken by python (which is a huge dependency but not another huge dependency).
> Though.. what if I told you the above four lines could be replaced by just... one?
I'd reject the pull request. Bash is already bad as programming language (the goodness of language for long code is inversely proportional to how nice it is for shell one-liners), this is just turning "bad" into "line noise"
If your bash script takes more than one screen, rewrite it in Python, hell, rewrite it in Perl, even that's better
The "if !" syntax isn't available on old enough shells.
Probably not an issue for most people in 2026 -- you have to back pretty far for it to be missing. Technically, though, "if x; then :; else" is more portable.
First of all, I must salute you on the joke; well put haha!
---
It is a (contrived) example of usage where a command is required and `:` can fill in the blanks. There are certainly scenarios where negating an expression becomes harder than doing the "dumb" way, and I for one has written code where `:` can be used as a placeholder meaning "fill this in later" or equivalent.
With that said, 100% agree with you that in actual "production" code - there is always a cleaner way.
It's almost but not quite in the article. And you don't necessarily always want this. It depends if you want else-cmds to run only when condition fails, or when either condition or then-cmds fails.
You could write the word true instead of :, coincidentally showing that : never really was a no-op in the first place. It's so not-no-op that there is even an entire external executable /bin/true to do the same job.
You misunderstand. I didnt need an explanation of how they worked. Im refuting that they are a good practice.
I was complaining of the "cleverness" of them, when working or analyzing a codebase with people swapping from if/else loops to trinaries.
I prefer clarity and a bit more verbosity than a per5-ism. If that means a 5 line loop that I can easily follow, then so be it. 1 liners that effectively say "lookie at me im a l33t developer" are a very bad code smell, and make maintenance harder each cycle of more cleverness.
This is an excellent article that helps people decide
against writing shell scripts. I abandoned doing so
shortly after I switched to linux. Since then I was
also using ruby. I still do not understand why people
would prefer shell scripts over ruby (or python). On
systems without ruby or python, one may see a benefit
in using shell scripts; other than that I fail to see
why shell scripts are necessary.
Shell scripts simply suck for many reason. They are
ugly, verbose, convoluted, outright stupid too such
as argument passing into functions. Then there is
straight up retarded stuff such as case/esac. Whoever
came up with that was clearly an incompetent language
designer.
I like your 2026 perspective. Objectively, the posix shell syntax is genuinely bad, even though the overall system that it enabled is amazing. It's a very old system that took off because it solved a problem - history is kind to it because of that context. Though, awk is around the same age and provides a proof-by-existence that posix shell syntax didn't have to be make these mistakes.
Articles like this are fun but they all come from posix shell syntax being fundamentally bad for scripting/programming. All the piping stuff is great, of course. And the overall ecosystem is great. But the interpretation of the script itself working by a series of string substitutions is a mechanism we wouldn't accept in a regular programming language. And there's no excuse for it really, except that shell syntax is really, really old.
For example, what does `$foo` mean in shell syntax? In any reasonable language (perl or powershell, for example, or python if you drop the `$`), it's an expression that evaluates to whatever value's inside that variable. In shell, `$foo` isn't an expression in that sense, and what it does depends on what's inside it via a variety of string substitution rules.
This is the main reason we have arcane articles like this.
That said, nice article.
I feel the need to push back on this perspective.
Old doesn't imply arcane. It instead can (and does, in this case) mean that it won out over the course of decades against other less worthy alternatives.
Although you can write "programs" in sh, the shell syntax was never meant to be anything like a systems or application programming language. It was meant to efficiently automate systems tasks. It is kind of like the fork lift of computing. It does it's job very well, is bad at doing anything else, yet no other kind of vehicle can do it's job nearly as efficiently.
Comparing the shell to C, Go, Rust, Python, or JavaScript is crazypants. It has a different job than those, so of course it will look and feel different!
There are plenty of alternative shells that don't suffer these problems. Posix syntax didn't win because it was better - it's the classic 'worse is better'.
> Old doesn't imply arcane.
Indeed so. For example, awk is about the same age but doesn't suffer the same problems.
> the shell syntax was never meant to be anything like a systems or application programming language
A programmable shell is not only like a programming language, it is a programming language. Systems tasks have some specialised requirements but, overall, scripting is programming. There is nothing in the problem domain that requires the posix string-substitution problems.
If fish or oil had existed back then, they would likely have won, since they are better.
2 replies →
I think Bash won as Bityard says above, in the sense that COBOL won, which was because it was everywhere (approximately) and then when another system was being built to do something in COBOL’s niche COBOL was the obvious choice. Once it was sufficiently established it won by inertia and absolutely not because it was the better language. Or maybe it was the better language in its time, but it certainly is not now. And note I’m only talking about COBOL’s niche, of mainframe-based processing in the banking, insurance, state and federal payroll and similar areas.
1 reply →
> It was meant to efficiently automate systems tasks.
Shell was designed to launch processes easily. Unix philosophy pushed to make every function an application. And that's ultimately what shell "syntax" is.
There's shockingly little syntax in the shell. A lot of what we think of as "shell syntax" is actually full blown (standard) programs which the shell is launching. My favorite of which is `[`.
What makes shell unique vs all other programming languages is it puts launching processes first. It has no friction there because if it did, it wouldn't work.
This is also what I think is gross about shell. Every other scripting language has at least some friction when launching other processes and that's usually a good thing IMO.
One of these are not like the other. Python is perfectly serviceable as a shell, to the extent that one could largely replace bash with the Python REPL if they wanted.
9 replies →
I disagree. I see shell syntax as an “inversion” of normal quoting rules. Normally quoting is the default but to achieve anything special you have to use an escape. Conversely, in shell, quoting is not necessary but any special character will likely have unwanted behaviour unless you quote.
This is handy for quick scripting as a natural extension of a command line interface. A counterexample might be an equivalent script using the Python subprocess module. For a simple script, the required extra quoting overhead is not worth the trouble. For something complicated it’s essential and a shell script is no longer suitable. For something in the middle, a carefully crafted shell script (eg. one that passes shellcheck) may be the most suitable solution depending on the situation.
Both cases have their place. It’s about using the right tool for the job.
> fundamentally bad for scripting/programming […] a mechanism we wouldn’t accept in a regular programming language
I don’t agree with this assumption that shell should use mechanisms we accept in “regular” programming languages, whatever that means. And why not the other way around? Why do we accept mechanisms in programming that we wouldn’t accept in a shell? There are layers of assumption in your assumption.
I also disagree with lumping scripting and programming together. Those are two different activities, and shell is better at scripting than programming, and it’s better at scripting than programming languages.
Try using python as your shell, and you’ll find out that python is fundamentally bad as a shell REPL. The shell is necessarily different from python or C++ or <your_favorite_programming_language>, starting with the ability to run command lines by typing them, with zero extra syntax. That is something regular programming languages can’t do, and it leads directly to the need for string substitution rules.
If you want a shell made out of a good programming language… make one! People have been thinking about this for decades and nobody has come up with a good one, because shell languages like bash are better at shelling than programming languages.
a bunch of happy xonsh users disagree
1 reply →
Your `$foo` example confused me at first because I thought the backticks are part of the example.
Sorry, the backticks were merely meant to delimit the code.
I will immediately remove all the shell scripts on my system based on this comment alone.
Posted from my phone since my system came crashing down.
Heh, I like that. We are indeed stuck with posix shell syntax; I didn't claim we weren't.
1 reply →
Yes a large percentage of bash scripts fails if you feed them filenames with spaces or other special characters. Or a large amount of filenames near the Unix maximum commandline length.
Bash et al are great for command line use, but produce a situation of really bad engineering hygiene when used in scripts.
Do not use. Stay away. Bash et al considered harmful. Red flags on job interview when referenced.
Whilst I agree that Bash makes it far too easy to do it wrong, it's not too tricky to write scripts that don't have those issues.
Possibly the most important tip is to use ShellCheck (https://www.shellcheck.net/) as a linter on your scripts and work through the various warnings to eliminate them. For the rare times that ShellCheck is overly cautious, you can remove a specific warning with a comment before the offending line such as
Also, take time to look through Greg's Wiki as it is possibly the best Bash resource that discusses the various pitfalls: https://mywiki.wooledge.org/BashPitfalls
As a Bash script writer, I disagree with your "red flag" interpretation as that'll mean that you'll end up with poorly written Bash scripts along with a hotch-potch of different tools to achieve the functionality (e.g. different versions of Python required for older scripts and newer ones). Knowing Bash is also very useful for writing Dockerfiles.
Well that's exciting. I learned a lot of uses for ":" today.
However, the only one I already knew...
I used to do that until I learned of
It's in the POSIX standard so it's not just a bashism: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...
> If the pipeline does not begin with the "!" reserved word, the exit status shall be the exit status of the last command specified in the pipeline. Otherwise, the exit status shall be the logical NOT of the exit status of the last command
https://github.com/anordal/shellharden/blob/master/how_to_do...
Do you have a real-world example where we would like to preserve `$?` after the `if` (which `if` specifically branches on the result code of the command it tests)? Where we want to use the actual return code that the `if` did not handle itself.
I am asking because I have never needed preserving the `$?` outside of an `if`, nor have I ever seen this in somebody else's scripts, and I'm curious.
TL;DR: The old-school empty branch preserves $?, while logical negation doesn’t. Sure I guess, but this is not always relevant.
I once created a set of shell functions which I wanted to have a docstring-like functionality. The solution I came up with was to have each shell function start with the : command with a string as an argument. Since : is an actual command, not a comment, it was preserved as part of the function, and could be extracted at runtime using relatively simple parsing to do introspection.
Example:
(Repost of <https://news.ycombinator.com/item?id=29152308>)
I am not a huge fan of most of these, but a few do seem useful.
I wouldn't use this because I would want to give $1 a name for the rest of the script, so I would assign. But it can be a nice way to give a clear error for missing required environment variables.
Many of the others (like truncating files) are probably more clearly written with dedicated commands, but may come in useful if you are going to extreme lengths to avoid dependencies outside of the shell.
Yo, author here.
You are very much correct and I 100% agree with you, I have updated the first example to include a snippet where a proper env-var is used to show of the automatic diagnostic.
Thanks for your feedback, much much appreciated!
For a short-lived script, the `${1:?missing argument}` stuff may be useful, but usually, I want to print some longer usage or help text in the error case.
I usually go with something like this:
The `${1:-}` in this case evaluates to an empty string if `$1` is not set, but `shift` fails when there is no argument to remove.
Agreed; author does note
> Why use the null-command when I could do VAR=${VAR:-default-value}?
and points out it's one less thing to typo, but that assumes the name is the same; I like e.g.
$1 might also just be the first parameter for a function call, so you might not need that parameter to have a specific/readable name if the function is not very complex or long. I have plenty in my dot-files that are just one-line functions that don't need to be three-line functions just to give the single parameter it accepts a name.
"Shortcuts" like :? that tie together two unrelated things (checking a variable for a condition, expanding the value of that variable) drive me up the wall. It makes expressing just one of the two things harder than expressing both, leading to contortions when you want just one, like this use of :.
I use this for setting overridable defaults (this should hopefully actually be in the article I haven't read yet)
: ${DEBUG:=false}
debug is not only false, but garanteed to be set to something so that elsewhere the places that use it don't need extra syntax and checking in case it's empty, and yet you can just set it from the parent environment to override without touching the script or adding a commandline args parser.
And do me, reading this is almost as easy on the brain as just plain DEBUG=true.
Even if you aren't familiar with the extra syntax, like you're someone else in the future who needs to look at it, the two important words pop out, and probably don't mean NOT because whatever those extra bits mean, there's no !. So you get the idea well enough & cruise on.
...[edit] yep it's in the article. but one thing isn't, exactly
In other words, maybe you should have figured out some other way to write the whole construct, but IF you want to write the if/else this way maybe for just readability or organization, and you want the else-block to only hinge on the initial condition and not also on whatever the then-block might do, then you need a safety-true at the end of the then-block.
[dead]
I use the colon as EDITOR with Git when I want to do an interactive rebase combined with auto squash without having to edit the todo list.
I have an alias[1] for that which I call a quick interactive rebase:
[1]: https://github.com/fphilipe/dotfiles/blob/94f2ff70bade070694...
That seems unsafe. It assumes that the editor is started via a shell command, and not executed directly. But there is no actual binary /usr/bin/: but there is a /usr/bin/true; I would use that instead.
It assumes git works the way that git works, and that git will not break backwards compatibility in the future. I think the same can be said of any alias in git.
1 reply →
damn, nice one — definitely gonna use this myself (or variations thereof)!
nice dotfile
I actually did absolutely need it recently when I golfed together a shell script that is simultaneously a valid YAML file [0]. Sometimes having no-op tokens is nice!
[0]: https://domi.work/blog/posts/compose_polyglot/
My personal favourite use of the colon command is what I've written about some years ago: https://johannes.truschnigg.info/writing/2021-12_colodebug/
Why take a perfectly readable if-statement and turn it into something, 99.9% of people would need to lookup. Concise != better. You can make it one line with:
This question seems to pop up all over the place, so I took some time to answer it here:
https://refp.se/articles/your-shell-and-the-magic-colon#why-...
the people writing it are doing it for themselves probably. i wouldn’t expect it to survive a code review. the necessity of using esoteric bash features or syntax is a pretty good smell that you should be using something else imo.
different strokes of course.
There’s a good chance that the people writing it will probably forget what it does in six months too :P
I believe you missed a semicolon after `exit 1` and before `}`. `}` closes the opening `{` only at the start of a command (POSIX rules; don't remember now if bash recognizes it when it's just a command argument).
I'm just too used to how parsing works in Zsh. For bash and sh, there is indeed should be a semicolon.
In this specific case, echo can't fail to there's no need for any of that:
1 reply →
Off-topic, but I am reminded of Larry's First and Second Laws of Language Redesign (which Larry Wall discovered/stated when he designed Perl 6, which is now Raku):
This "hidden knowledge" is fun to read but pain to remember and use, especially when working with multi-platform environments //
Switched from bash to plain python scripts for shell stuff everywhere several years ago, and never looked back into bash zoo anymore. Stable syntax across Win/Mac/Linux, no bash/zsh/msys2 obscure differences, normal errors and Clause writes scaffolds quick and flawless anyway
I guess if you're never calling out to any third-party python libraries, that can work.
I'd never call Python "portable" though if you have outside dependencies.
I find Python a terrible bash replacement.
Calling all the command line tools is cumbersome, the Python cloud APIs are verbose compared to the cli tools and often have very different concepts for how they interact with services.
But I agree that there is a need for a better bash and zsh is nice but not the upgrade that is needed
Nice one! I love those weird bash tricks.
Some of the examples here are interesting, but they show parameter substitution more than colon itself: https://tldp.org/LDP/abs/html/parameter-substitution.html
In small scopes, I tend to inline the `:?` validation inside the arg of the command. `echo "${1:? first param required}"`
Another usecase is to use colon in the body of a while loop, while doing work in the condition of the loop.
Gives you the "do X while it succeeds. stop when it returns non-0" semantics.
I've also written about this and other bash tricks over the years in https://github.com/kidd/scripting-field-guide/blob/master/bo.... You might like them :)
I hate weird bash tricks with a passion.
The extent to which the script author gets to feel smart and efficient is exactly the extent to which the future reader of the script gets to feel like an idiot.
So true. I think most of us who learned "advanced" bash is after being tired of feeling like an idiot looking at long bash scripts.
Usually, those scripts are written by people who didn't learn the "weird tricks", and use global variables (they didn't learn that `local` creates local variables), for example.
In bash, many times the "dumb way" takes 10 times more lines, and it's probably buggier than the "smart tricky way". So, in the end it's your choice what side you pick, and when to stop. Or, use a "real programming language" (I hate when people say this LOL)
After a while, you get a bit of stockholm syndrome, and you're just fine with bash as an orchestration language, and lean into unix principles. I accept its limitations, and I think it keeps me on my toes as it repels bloat.
idk if you were expecting an answer, but bash lives in this liminal space in the tower of languages that is worth a thought.
The only upside of weird bash tricks is there's so much bash out there that the LLMs are really well trained on it - so you won't have to read the script, just the tests.
life is way too short to deal with this nightmare of a language and its 50000 footguns for anything longer than a 2 line script, especially in the age of LLMs. Just write a python/TS/any real language script instead. Bash is great for the command line, it should be limited to use there.
It turns out that LLMs are really good at writing bash too. even perl! maybe we should rethink some of these lost bits because we no longer need to worry about the arcane parts.
Define "really good?" I don't think I've ever had them produce a script that I didn't need to correct in some way. They can write it, that's true, but we still need to be able to read it.
4 replies →
Bash scripts requires "set -eu" at minimum, when written by LLM or human.
It's the only language terrible enough to make the default behavior ignore undefined variables, commands, and execution errors, and happily continue executing whatever was produced by me smashing my hands on the keyboard, until the end of the file, while returning an exit code of 0, claiming complete success.
8 replies →
I have seen them write massively more complex bash scripts than any normal person would ever attempt, but the failure rate is absurdly high compared to when they write sane typed languages.
I have some absolutely amazing bash scripts that I would never have contemplated making myself in bash. And they'd only have been somewhat better in python. Claude for the win!
An LLM taught me how to use the shell colon. Shell scripts are short enough that it's been worth my time to edit the AI's work.
I find LLM's too fall into the trap of writing a bash script for a task that clearly needs to be implemented in an Actual Language with Real Data Structures. For example, ask an LLM to bring up a SQL server with some schema + data preload step, and it will write a profoundly long bash script to do that task, every time.
my experience as well. claude produces functional but over-engineered scripts fairly frequently with often very questionably useful safety checks, especially for powershell.
a common tell of ai generated powershell is a script that has dedicated functions to check types, often via several methods, and happily prints the output of the checks to shell. i do not get why it does this but it often adds dozens of lines that really serve no purpose but to make the shell output look fancy
2 replies →
Try being on linux and using zsh, it constantly fucks up scripts. Having it run commands via SSH into a windows machine is even more a comedy of errors.
I have to put in every claude.md that the shell is zsh. It's reached the point of annoyance I might just go back to bash.
4 replies →
I think they've learnt from decades of humans using Bash for tasks that clearly need to be implemented in an Actual Language.
The knowledge that Bash is awful and should be avoided as much as possible is surprisingly and disappointingly rare.
Python startup time is a problem. I don't know about ts but I bet it's worse.
TS compiles to JS, so when you actually execute a script, you’re just starting Node, Deno, etc. with a JS file. Any of those are probably faster to start than Python. Deno in particular starts in around 15-60 milliseconds. Even if it has to compile the Typescript - which is a one-time operation if the script isn’t changing - it still typically starts in under 250ms.
What other language is it as easy, fast, and painless to work with file contents? I can toss data down and lift it back up with some <> symbols and a path, none of Python's "errrm, how would you like to open that file " handholding. And why the hell would I want to go learn TypeScript!?!?
Sometimes I feel like the people on HN repeating the trope "uhhh if you need ____ go learn a real language!" are just bad at writing shell scripts.
Just use Fish
Or better yet Nushell
> 50000 footguns
(...)
> especially in the age of LLMs
Funny way of putting it.
But I think you are right still :)
I have probably the worst use case, but I like it. I have a very specifically structured ZDOTDIR, and I write everything in a way that is self-documenting.
After a while, you probably know more commands and utilities than you know what to do with, and you'll forget they exist when you need them. In order to not waste time looking for a program for a particular and infrequent purpose, I create "do nothing" aliases like `: alias f3probe` so I can realize I just forgot I already have something for it. Nice predictable pattern to grep with `^: alias` to look through all of these.
What if there was a less cryptic way to have mandatory arguments, something harder to get wrong, like
And then:
Or non interactively:
PowerShell needs to become more ubiquitous. There are so many things about it that once considered in greater detail make so much sense. Parameter setup, typing, and naming. Flow control. Object-oriented scripting. Built-in parsing for recursive data types like JSON, HTML, XML.
And in my opinion, the most slept-on: the fact it runs on the CLR and direct access to .NET objects and types which means access to P/Invoke and thence the Windows API. One can write business logic in the fast language and write a nice CLI wrapper around that in the natural shell language, and not worry about painful FFI unlike everyone else trying to fit Python or Bash into whatever world they're using.
The typical counter to this will be: PowerShell is verbose, PowerShell used `curl` as an alias to Invoke-WebRequest instead of the Real Thing™. Neither are real arguments.
I'm curious, have any good examples of projects in the ~1K LOC range of PowerShell, so I could get a taste for how something like that would look like? Beyond that I tend to go for "real" programming languages, or just start with Babashka, but still curious to see projects other PowerShell fans would consider "well written/designed" :)
1 reply →
DUDE!
I wrote my first real `.ps1` the other day for auto-installing all dependencies needed to run a `gitea`-runner on windows for windows builds; powershell - felt like the lover I never had. And the documentation in readable comments at the top that just.. generates usable docs? Damn, damn, damn.
There is a part of me that low-key wanna try that as my daily driver for a week or two. But with that said, I'm a zsh vi-mode guy - always have been, always will be.. but I'd happily take powershell on a romantic getaway every once in a while!
An attempt was made, but just run `help get-item` in powershell and tell me whether it sparks you with joy.
PowerShell is a bad choice for the system shell because it's too complex. I'm not trying to justify all the quirks of Unix Shell, but minimalism is a very important feature of such a tool.
Another important feature of a tool like this is the ability to tolerate errors: I can't imagine a Linux today that would be able to even boot if the shell was extra pedantic about errors. A lot of mostly irrelevant things routinely fail on boot and during normal operation. Stamping them all out is an arduous... well, basically, an impossible task for practical purposes where releases are expected to come on time, where users may manipulate configuration in gazzilions of unpredictable ways.
PowerShell is just another language in the same box with Python, Perl, Ruby and many like that. It's not a good language, if you decided to reach for that box. Probably not the worst either.
System shell, however, isn't meant for writing entire applications. Writing applications with elaborate command-line interface should be left to languages that can properly address this problem. PowerShell is trying to be there, but it doesn't hold a candle to its "older brothers" who can, indeed, design a very robust command-line interface, often using a dedicated library for it.
PowerShell appeals to the novice crowd who are very enthusiastic about automatic checks in their code: the benefits are on the surface, the downsides are difficult to assess. This is in line with other Microsoft software products / languages which target novice programmers by implementing as many as possible of the highly-advertised features without regard to the overall usefulness of the product (think about C# or MS Office suit etc.)
> System shell, however, isn't meant for writing entire applications.
Programmable shells are, in fact, for programming. That posix shell syntax makes it impractical for meaningfully large scripts is something you just accept. And, realistically, it is something you have to accept.
Minimalism isn't why posix shell syntax is bad. For example, awk and jq are minimal but don't make the same mistakes.
Powershell was never designed to be your system shell (Windows doesn't really rely on one) but the main reason it can't be the shell is startup time, which rules it out of wrapper scripts. Its having lots of features might contribute to that but it's not the problem per se (being built on .net is the reason - that and startup time not really mattering on windows). Its lacking the syntax problems of posix shells is very much not a problem. Powershell's error handling is just fine, by the way.
The startup time of even pwsh is too slow to be usable in many scenarios.
PowerShell's not going to be the system shell on your *nix box but it wasn't designed to be. But your posix shell isn't the system shell because it's better. It's pretty much the ultimate expression of worse is better.
That's why there's an industry of alternatives without the faults, such as oil, elvish, fish, nushell. And why awk is so popular.
1 reply →
> Another important feature of a tool like this is the ability to tolerate errors: I can't imagine a Linux today that would be able to even boot if the shell was extra pedantic about errors. A lot of mostly irrelevant things routinely fail on boot and during normal operation. Stamping them all out is an arduous...
You seem to be implying PowerShell aborts on any error. That's not the case:
https://learn.microsoft.com/en-us/powershell/module/microsof...
1 reply →
My main issue with attempting to replace Bash with other shells is that they're often not installed by default on different Linux flavours and they don't/won't have the same longevity as Bash. Try running PowerShell on a 20 year old Linux system or maybe find out if there's some version issue with running an old script 20 years in the future.
Well said, esp. the last paragraph about the Microsoft strategy.
the truncation is a poor example.
- it creates the file if it does not exist, not merely truncate. as a tutorial kind of blog post this incomplete description matters IMO.
- it would work the same without the colon (similar for default variable assignment examples). we generally strive not to have "extra" things, like useless use of cat.
- educationally it's useful to demonstrate that redirection, like parameter expansion, works before the command executes (the null command in this case), but the article doesn't explain that at all!
otherwise i <3 this article. some uses of colon i had never thought of or seen before. like file truncation, not sure i'd use them but it was cool to see them.
Hey jiveturkey, author here!
I agree with you, and for what it's worth the truncation snippet is very much tongue-in-cheek much like `( : >> output ) && echo "is writable"`.
I wasn't expecting anyone to actually use these in Prod, rather I aimed to show what can be done with a command designed to.. do nothing (crazy).
Happy you enjoyed the article, and thank you!
Whats the advantage of the colon for the truncation example
versus
I've done quick and dirty, interactive truncation like the former for many years, no colon. But I would not use it in scripts
According to https://www.in-ulm.de/~mascheck/bourne/ SVR4 (1989) had a bug when using this method in a for or while loop and this bug showed up in a SunOS 5 variant, too
Apparently, early in the shell's evolution, : was used as a comment marker before # was added
Single quotes could be used to prevent undesired behaviour
System III (1981) also had a bug when using : as a substitute for true
returned 1 instead of 0
I will try to internalise this knowledge, but I find remembering terse or deep obscure syntax so hard.
I often use : to set default values of configuration envs. e.g, in my dotfiles bootstrap script I have:
Which will use $DOTFILES_PATH value if it's set, otherwise it's going to be $HOME/.dotfiles
I always wanted to learn more about scripting. But today I am not as passionate as before because LLMs write working scripts most of the time. I am wondering if it is true for most programming techniques and quirks. Are we going to write code to solve low level problems?
The other day there was a blog post about learning SIMD. I think in future "programmers" will just nag about the speed of the program and the coding assistant will eventually introduce SIMD to the source code.
It is a little sad but we have to go with the current if we want to survive.
I still think that learning the shell environment makes a lot of sense these days. But mainly for interactive use. It's amazing the scripts your LLM can make you with, say, fd and fzf.
I can see crafting line by line when I’m retired or for a personal project. It’s eventually going to be like the people now who still operate restored manual linotype machines. “Look here I have the last functioning one in the midwest. See how the plates work…”
That legitimately sounds like the dream. Maybe this is just the universe's way of giving us our train set?
Good to know, but looks less readable than `if` example.
> ( : < dataset.json ) && echo YES # is dataset.json readable?
The subshell execution parentheses and the colon are superfluous here, just:
Redirections do not require a colon command to hang off of, and there is no need to fork a subshell to execute such a command.
> ( : >> result.json ) && echo YES # is result.json writable?
As a go-to idiom for a writability test, it gives me pause. If the file didn't exist, we created a zero-length one. That might be okay if we are going to write to it anyway as the next action.
If we are testing because we intend to overwrite it, why not just "> result.json" (which is by itself an idiom for truncating a file to zero length).
When would we every do this? Maybe before some command which takes the file name as a destination file argument rather than using output redirection, and which performs a lengthy computation before trying to open the file for writing. We can catch the permission error early.
I don't think I've ever coded such a test; normally you just do the operation that writes to the file and let that fail.
In POSIX C, there is a function access() for doing these kinds of tests. But it has a special purpose: it is meant to be used by a setuid root process to perform a permission test as if it were the real user/group (the one which elevated privilege to root). I.e. it's not can we do this operation, but should we do this operation (would we still be allowed, if we dropped privileges back to the original user).
They are NOT superflous, and all you need to prove it is `zsh` (but there are others that follow suit in similar fashions):
So, if you want something that "everyone can use" without going into details about the difference between commonly used shells.. you'd use the null-command.
---
and given that we use the null-command, it _WILL_ behave different with or without subshell.. and all you need is `bash --posix` to prove it:
The output above is not truncated, `no-subshell.sh` will stop executing due to the broken read.
---
One should never trust things just because they are written, but that also applies to comments on HN. Originally when I read your message I actually thought I made a mistake, I was very close to writing an apology comment and adding a note to the blog post, but not close enough - I had to test it again.
I'm thankful for the watchful eyes and scrutiny when reading things online, that's good - keep it up, but your message is factually wrong - on so many levels.
> : < missing.json; echo AFTER # <- this will not
This is required by POSIX in the section "Consequences of Shell Errors". If a redirection error occurs in a special built-in command, a non-interactive shell is required to exit.
The bare redirection without : does not have this problem:
Therefore, in a POSIX-conforming shell, we do not need to wrap it in a subshell.
> but your message is factually wrong - on so many levels.
How many? Can you count the levels? I didn't know that in zsh "< file" dumps to standard output, which is suppressed by :. If I used Zsh, I would know that sort of thing.
It’s very common that the most upvoted comment on a post is a mistaken correction. People seem to love comments seemingly proving that the post is wrong without actually checking anything. And once it has enough upvotes a big discussion may follow up with personal attacks on the author or other questionable comments, while the people trying to point out that the post actually is accurate get either buried under the critic storm or downvoted to oblivion because people just don’t like the truth. This happens in every controversial topic.
One of the reasons I stopped writing.
4 replies →
Maybe zsh still reads the entire file and copies it to /dev/null, which would be a severe performance problem for large files.
The colon by itself, without the subprocess, also prevents the dumping to stdout:
However, with the subprocess parentheses, we can redirect the nonexistence diagnostic to /dev/null, which I don't see mentioned or exemplified in the article:
The normal way of testing whether something exists and is readable that you would actually use in production script looks more like this:
so we are talking about obfuscated coding.
I would say that "if you want something everyone can use", the -r test would be the first candidate.
BTW, why not bring up the strawman of tcsh?
Yes, if we want an obfuscated readability test which works literally in any shell that is currently still in deployment in systems that permit new scripts to be installed and run, it looks like do need that colon.
However, fish doesn't like &&:
Does it support test -r?
Not parentheses though:
We clearly have to restrict the idea of what "everyone can use" to POSIX-like shells; there is no getting around shell differences absolutely, other than for perhaps trivial command invocations.
2 replies →
the colon does nothing, which makes it the only bash command an LLM can't over-engineer
One of the things I do with : is infinite while-loops, like:
Maybe to mainstream to make the cut!?
I prefer using "while true;" as that's easier to read. Arguably, it's less efficient due to calling the external "true".
Strangely though, I much prefer using the builtin "printf" to replace "echo" and "date", though the "date" usage is harder to read.
I've read through all of the examples in the article & they all seem to serve to sole purpose of turning readable multiple line code into one-liners.
One-liners are a cool little artifact of early shell culture & are sometimes still useful today if they're short to avoid the readability problems of `/` when copy pasting a quick shell command to run, but they have no place in scripts.
None of this seems useful to me.
> if you are like me and prefer less typing (gotta go fast)
Yeah, no.
Bash scripts should only be used for quick and dirty tasks where brevity is a major benefit. Don’t pretend bash can be readable and maintainable. If you want that use another language that sacrifices brevity for clarity.
First I agree that bash scripts should only be used for quick or small (& ultimately mostly personal unshared) tasks. There's absolutely no need for it to be either "dirty", & as mentioned in my post, brevity has no material benefit in this context.
I want my personal local utility/productivity scripts to be readable: quick to write & quick to modify on the fly. Brevity doesn't help here - wpm optimises for natural language typing & that translates better to idiomatic logical block structures than to symbol-heavy one-liners.
I also want the same for the small bash snippets in my CI jobs - this is a particular example where brevity is actively bad: this encourages folk to inline their bash snippets in yaml (no syntax highlighting & unlintable) when they should be packaged in script files in CI directories.
1 reply →
there are some early unix tapes floating around, and in those early shells, i'm fairly certain the colon was one of only two special-cased code paths after the command line was parsed. does anyone recall more specifically?
I think colon at the start of a line was used for labels for goto statments.
https://www.in-ulm.de/~mascheck/bourne/PWB/goto.1.html
Is : the same as `true` ?
I was asking myself the same question and looked it up:
<https://pubs.opengroup.org/onlinepubs/9799919799/utilities/t...>
The most important differences seem to be:
* `:` is a builtin vs. `true` is an utility * passing arguments to `:` is safe but for `true` its not hence `:` is the right thing to use here.
In the Bash shell, true is a builtin. It may be a builtin for other POSIX-compliant shells. false is also a builtin.
The key difference is that : is required to be builtin. There are several good reasons for this. For example, syntax: the shell can single out this special character syntactically before searching $PATH.
Also, ":" is a disallowed or problematic character for certain filesystem types. If your shell attempted to omit this builtin, it could not always rely on an external command file by this name.
Lastly, people keep bringing up fish, but it is not a POSIX shell, so its similarities in syntax and operation are coincidental.
the fish shell manpage has ':' as an alias to 'true'.
I really hate bash because of its unreadable syntax, and this does not help it in any way.
We have some large bash scripts in my company, ~10,000 LOC spread across multiple files, all sourcing each other and what not. It is truly hard to read bash, which means it is truly hard to maintain bash, which means that when the one person knowing the bash scripts in your company goes away, you're in for some "fun".
My point is, these quirks are not useful, except for some bash enthusiasts.
Its not "unreadable" syntax, it's terse and unfamiliar. There are many languages with terse and unfamiliar syntax to me that I struggle to read, but only because I'm unfamiliar. In fact, most programming languages have unreadable syntax at first, save for maybe...AppleScript? Maybe Scratch. Even Python's famous readability isn't guaranteed, some of its functional programming facilities are just as arcane and unreadable as *sh.
I spend about 50% of time working time in the shell. I didn't find anything in TFA unreadable.
Absolutely!
Your company likely started using bash before python was ubiquitous.
I usually skip the get option builtin, and use : as...
while : ; do case "$1" in "") break;; -f|-foo) shift; whatever;; *) usage; exit 1;; esac done
For this... instead
Using : would be too much here.
For anything else including json etc. I usually go to duckdb. Awesome support, single file install, readable, easy to maintain.
Powershell on Linux or Unix? Just another huge dependency if you manage 1000s of machines, and good luck finding a Linux gal/guy wanting or able to touch pwsh without chemical grade gloves.
Yeah, I don't see pwsh finding a place in the ecosystem. Even oil or fish struggle. Their place was mostly taken by python (which is a huge dependency but not another huge dependency).
Thank you for this.
Also, I'll never use it.
Because a language feature that needs marketing is against readability, among those in my target audience who have not yet read the marketing.
I need my shell scripts to be long enough to explain to my audience exactly what they are doing.
> Though.. what if I told you the above four lines could be replaced by just... one?
I'd reject the pull request. Bash is already bad as programming language (the goodness of language for long code is inversely proportional to how nice it is for shell one-liners), this is just turning "bad" into "line noise"
If your bash script takes more than one screen, rewrite it in Python, hell, rewrite it in Perl, even that's better
Stupidly terse code is fun and a good laugh to show to people as a puzzle… it is the opposite in an application that relies on it.
I totally agree with you.
> the goodness of language for long code is inversely proportional to how nice it is for shell one-liners
Well, except for this bit. Oil or fish and awk are perfectly fine for one liners but don't repeat the mistakes of posix shell syntax.
Here's another tip I picked up last millenium: If you're using a GUI and like decorating your shell prompt with information, format it like this:
Then you can copy/paste entire lines of commands.
(Yes, this assumes you don't put naughty fragile stuff in your prompt. Buy you're smart enough not to do that.)
Prefer
over
Really? Colon is the appendix of the shell.
The "if !" syntax isn't available on old enough shells.
Probably not an issue for most people in 2026 -- you have to back pretty far for it to be missing. Technically, though, "if x; then :; else" is more portable.
Aside: hacker news doesn't do markdown code blocks, but instead uses 2 spaces before text;
Thanks, fixed
First of all, I must salute you on the joke; well put haha!
---
It is a (contrived) example of usage where a command is required and `:` can fill in the blanks. There are certainly scenarios where negating an expression becomes harder than doing the "dumb" way, and I for one has written code where `:` can be used as a placeholder meaning "fill this in later" or equivalent.
With that said, 100% agree with you that in actual "production" code - there is always a cleaner way.
https://www.gavi.org/vaccineswork/what-does-appendix-do-biol...
For decades I used bash and never knew and saw this syntax. And recently discovered that by a code generated by Claude.
And now I see this article. So I guess that it is a construct suddenly popularized by llm.
Cool!
Btw, can the sequence :? be called "reverse Elvis" ?
Maybe it's just the fact that I woke up 10 minutes ago, but the readability of this looks awful. Even more than just usual shell scripts.
ensure true
It's almost but not quite in the article. And you don't necessarily always want this. It depends if you want else-cmds to run only when condition fails, or when either condition or then-cmds fails.
You could write the word true instead of :, coincidentally showing that : never really was a no-op in the first place. It's so not-no-op that there is even an entire external executable /bin/true to do the same job.
interesting
Another fun use of colon:
:(){ :|:& };:
I've never liked cleverness in scripting when clarity costs effectively nothing.
This is one of those clever things, similar to people using perl5 use trinary expressions. Like, are you TRYING to make this obtuse and hard to read?
The trinary is semantically just an if expression though. Perl inherits C's problem that `if` can't be used in an expression context:
Because of this, Perl introduces a second syntax to plug the gap:
That expression is neither obtuse nor hard to read.
Something like the code below is hard to read:
It's equivalent to the following code that uses `if`:
But ideally you would want to write this:
Though, of course, perl5 doesn't allow that.
You misunderstand. I didnt need an explanation of how they worked. Im refuting that they are a good practice.
I was complaining of the "cleverness" of them, when working or analyzing a codebase with people swapping from if/else loops to trinaries.
I prefer clarity and a bit more verbosity than a per5-ism. If that means a 5 line loop that I can easily follow, then so be it. 1 liners that effectively say "lookie at me im a l33t developer" are a very bad code smell, and make maintenance harder each cycle of more cleverness.
1 reply →
[flagged]
[dead]
[dead]
[flagged]
This is an excellent article that helps people decide against writing shell scripts. I abandoned doing so shortly after I switched to linux. Since then I was also using ruby. I still do not understand why people would prefer shell scripts over ruby (or python). On systems without ruby or python, one may see a benefit in using shell scripts; other than that I fail to see why shell scripts are necessary.
Shell scripts simply suck for many reason. They are ugly, verbose, convoluted, outright stupid too such as argument passing into functions. Then there is straight up retarded stuff such as case/esac. Whoever came up with that was clearly an incompetent language designer.
I like your 2026 perspective. Objectively, the posix shell syntax is genuinely bad, even though the overall system that it enabled is amazing. It's a very old system that took off because it solved a problem - history is kind to it because of that context. Though, awk is around the same age and provides a proof-by-existence that posix shell syntax didn't have to be make these mistakes.
Mostly agree, but many times there is no python nor ruby on a system, whereas there may always be shell available for scripts like these.
Ahh yes, that Bourne guy is clearly incompetent....
https://en.wikipedia.org/wiki/Stephen_R._Bourne