Or you can just…not quote the tilde. Folks always seem to reflexively quote “strings” in Bash while not realizing that (almost) everything is a string and most strings are not quoted and it would be odd to do it (e.g. no one is doing `"ls" "-a" "foo"`).
Unless you're running a shell command from python. That was the first time I saw a command string broken down into "string" arguments for every thing like that.
In that case, you're not running a "shell command" from python, you're passing arguments to exec. A shell command would be a string interpreted by the shell, and you'd use that for shell syntax things like having the shell do variable interpolation or redirections as part of executing the command.
The Z shell's, C shell's, and others's syntaxes for setting the PATH environment variable via a shell array variable alias also does the tilde expansion.
path=( $path ~/bin )
It's worth noting, also, that the path and manpath settings in login.conf(5) expand leading tildes in individual search path items.
Not necessarily about tildes but about some of the craziness that can happen with bash at large orgs:
At a past job, I was trying to figure out what part of my basrhc was setting a particular environment variable. I assumed that it must be some kind of default installed in my user profile and/or inheriting from /etc/<something>.
I realized pretty quickly that my bashrc was importing some other files. Again, the assumption was that this would be one file deep in the import.
It turned out there were 10+ layers of import starting from an "ur-bashrc" and then layer upon layer of more and more imports to finally get to a user level profile.
I wish Linux distros would ship a "Terminal"/"CLI" program etc that is decoupled from the scripting language. Have a universal Path env var that isn't tied to a specific shell. Lets you execute cd commands, launch python/git/cargo/arbitrary applications etc, and have a good bookmark + autocomplete system. It feels like the conflation of scripting language + CLI application is the root of these complications and subtleties.
If you are using shell scripting (And prefer Bash etc over Python), you would keep using Bash/Fish/Zsh etc. If you are using the CLI to launch applications that don't have a GUI, navigate directories and perform file system operations, then you would use the plain terminal.
It is - this is why adding something to the Path is tricky on Linux. Or I should say, there is a mismatch between the common instruction of how to do it (export) vs what you have to do (e.g. edit bash config)
I was going to say “it always seems to work for me” then I saw “… actually works in Bash and Zsh, because …”.
Another Bash-ism I need to be careful not to use when trying to be portable.
It is worth noting that on a lot of systems /bin/sh isn't bash (or zsh) so if you want to rely on Bashisms (or just can't be bothered looking for them) be specific and use “#!/bin/bash” for you hashbang. On Debian and similar it is usually dash for instance.
I was expecting this article to be about skew between HOME environment variable and getpwent(3) (usually from /etc/passed) views of the home directory.
They can be different things. Suppose your user is foobar, ordinarily homed at /home/foobar. You can write HOME=/tmp/my-test-home. Then, ~/qux will become/tmp/my-test-home instead of the usual /home/foobar/qux. That's because bash and zsh use HOME to resolve ~.
Almost. If you write ~foobar/qux, you get /home/foobar/qux again because the ~-with-username syntax looks up the getpwent home directory not the environment one.
And of course language runtimes are all schizo about whether the "user home directory" API uses HOME or the getpwent database or whatever to determine the home directory.
It's a mess, TBH. It used to be useful to temporarily bind HOME to something else to do things like create isolated test environments. Now, because of the aforementioned schizo sprinkler of randomness in the environment, you're going to have a bad time if you don't keep HOME synced to getpwent home directory.
In most use cases, the bash scripts location is more important than $HOME. This is because it is resilient to changes in user in the session, and parent process current working location contexts. =3
Afaik it's habit to give system paths precedence so a malicious script can't shadow e.g. sudo and steal your password, escalating a local file write into root
So many headaches could be avoided if we only allowed `[A-Za-z0-9._-]` in paths. (Arguably, even `-` can be problematic.) Encoding issues, expansion, parameter separation, ... and I never saw a convincing case in favor of supporting anything else.
The language I grew up with does use non latin letters, I can manage. Then again, it's only four of them, so I get your point ... but I can dream, can't I?
I always joked about setting a custom keyboard layout to replace the space char with the underscore char specifically for avoiding spaces in file paths.
Or you can just…not quote the tilde. Folks always seem to reflexively quote “strings” in Bash while not realizing that (almost) everything is a string and most strings are not quoted and it would be odd to do it (e.g. no one is doing `"ls" "-a" "foo"`).
Or just stop using bash. It’s a terrible language to write and has tons of footguns.
For the things shell is good at (running other commands, pipes, and shuffling files), I have yet to find anything even close to as good.
Unfortunately half of its badness isn't isn't the shell itself, but the convention of how parameters are passed to processes on Unix systems.
3 replies →
The widwit answer: "Dont do X" (and nothing more)
The enlightened answer "You should use A, B or C for these reasons"
Even though bash is installed on many systems, I try to encourage people to use better designed shells that have less of these footguns :
Fish (https://fishshell.com/): No implicit word splitting : spaces in variables won't unexpectedly become separate arguments.
Zsh (I use this: https://ohmyz.sh/): Arrays start at 1 by default, but crucially, unquoted variables don't implicitly split into multiple arguments.
Nushell (https://www.nushell.sh/): Passes structured tables and records between commands : avoids fragile parsing of text with awk/grep.
I feel like this is common knowledge and should not be worth mentioning. But then it apparently is not common knowledge, as the article proves.
Have I run into this at some point?
I certainly have.
Have I learned to quote better and only where appropriate from it?
I certainly have.
Bourne compatible shells take a while to learn and require some experience. This won't change, but alternatives exist, with their own caveats.
Unless you're running a shell command from python. That was the first time I saw a command string broken down into "string" arguments for every thing like that.
In that case, you're not running a "shell command" from python, you're passing arguments to exec. A shell command would be a string interpreted by the shell, and you'd use that for shell syntax things like having the shell do variable interpolation or redirections as part of executing the command.
1 reply →
People quote both too often and too little.
GENERAL RULE
1. Double-quote dollar sign expressions, and nothing else.
2. Single-quote words with a literal special character, and nothing else.
---
I should point out that the author's example is NOT fixed by different quoting though.
Because tilde expansion only happens at the beginning of the word.
The Z shell's, C shell's, and others's syntaxes for setting the PATH environment variable via a shell array variable alias also does the tilde expansion.
It's worth noting, also, that the path and manpath settings in login.conf(5) expand leading tildes in individual search path items.
* https://man.freebsd.org/cgi/man.cgi?query=login.conf&sektion...
So putting the addition of things like ~/bin to PATH in /etc/login_conf and ~/.login_conf instead of shell scripts is another way to address it.
It's particularly handy when there are multiple login shells in use.
* http://jdebp.uk./FGA/BSDs-for-Linux-users/login-conf.html
The trick is to quote explicitly and correctly. “ and ‘ are different.
why would you use smart quotes in a terminal like that?
4 replies →
I never use tilde in scripts, only as a convenience when typing commands interactively.
$HOME otherwise, which still has gotchas but they are the same as any other environment variable.
Not necessarily about tildes but about some of the craziness that can happen with bash at large orgs:
At a past job, I was trying to figure out what part of my basrhc was setting a particular environment variable. I assumed that it must be some kind of default installed in my user profile and/or inheriting from /etc/<something>.
I realized pretty quickly that my bashrc was importing some other files. Again, the assumption was that this would be one file deep in the import.
It turned out there were 10+ layers of import starting from an "ur-bashrc" and then layer upon layer of more and more imports to finally get to a user level profile.
It was so convoluted that I was going nuts until I found this Stack Exchange post: https://unix.stackexchange.com/questions/813/how-to-determin...
It turns on "tracing" for bash imports so that you can then narrow down on where the env variable is getting set.
Kind of seems like you should have noticed this by your ~/.local/bin PATH not working?
I wish Linux distros would ship a "Terminal"/"CLI" program etc that is decoupled from the scripting language. Have a universal Path env var that isn't tied to a specific shell. Lets you execute cd commands, launch python/git/cargo/arbitrary applications etc, and have a good bookmark + autocomplete system. It feels like the conflation of scripting language + CLI application is the root of these complications and subtleties.
If you are using shell scripting (And prefer Bash etc over Python), you would keep using Bash/Fish/Zsh etc. If you are using the CLI to launch applications that don't have a GUI, navigate directories and perform file system operations, then you would use the plain terminal.
$PATH is not tied to any specific shell. And what you're describing is a shell, so I'm not sure how it's any different from existing solutions.
It is - this is why adding something to the Path is tricky on Linux. Or I should say, there is a mismatch between the common instruction of how to do it (export) vs what you have to do (e.g. edit bash config)
I always thought that there should be a proc style file system for simlinks to user specific data.
I'm not sure why this can't be done. Security people will have a million reasons, I guess it's possible one of them might be valid.
IIUC, you're describing an extremely restricted (some would say underpowered) shell.
It sounds like you could make it yourself in ~10 lines of Python or bash. I don't see it catching on, though.
Juste create a file ~ which is a symlink to home :)
Big brain time here
I was going to say “it always seems to work for me” then I saw “… actually works in Bash and Zsh, because …”.
Another Bash-ism I need to be careful not to use when trying to be portable.
It is worth noting that on a lot of systems /bin/sh isn't bash (or zsh) so if you want to rely on Bashisms (or just can't be bothered looking for them) be specific and use “#!/bin/bash” for you hashbang. On Debian and similar it is usually dash for instance.
I was expecting this article to be about skew between HOME environment variable and getpwent(3) (usually from /etc/passed) views of the home directory.
They can be different things. Suppose your user is foobar, ordinarily homed at /home/foobar. You can write HOME=/tmp/my-test-home. Then, ~/qux will become/tmp/my-test-home instead of the usual /home/foobar/qux. That's because bash and zsh use HOME to resolve ~.
Almost. If you write ~foobar/qux, you get /home/foobar/qux again because the ~-with-username syntax looks up the getpwent home directory not the environment one.
And of course language runtimes are all schizo about whether the "user home directory" API uses HOME or the getpwent database or whatever to determine the home directory.
It's a mess, TBH. It used to be useful to temporarily bind HOME to something else to do things like create isolated test environments. Now, because of the aforementioned schizo sprinkler of randomness in the environment, you're going to have a bad time if you don't keep HOME synced to getpwent home directory.
I should have read the docs before getting ~/ tattooed on my wrist.
Reminds me of early days of Cursor when it decided it would be a good idea to create a directory named "~" in my repo root!
That was a scary mistake to unwind!
Bash footgun #238503.
https://en.wikipedia.org/wiki/The_UNIX-HATERS_Handbook
That's standard POSIX.
Tilde expands when at the beginning of an unquoted word.
Pretty straightforward.
---
Bash has a few extra.
In most use cases, the bash scripts location is more important than $HOME. This is because it is resilient to changes in user in the session, and parent process current working location contexts. =3
scriptPath=$(/usr/bin/realpath "${BASH_SOURCE[0]}")
localPath=$(/usr/bin/dirname "$scriptPath" )
/usr/bin/echo "localPath = '$localPath'"
well.. it's in zsh (and probably any other *sh) too
In Zsh, you can set PATH with path=(~/.local/bin $path). There, it works as shell expansion works.
1 reply →
bad example in the article:
better:
Afaik it's habit to give system paths precedence so a malicious script can't shadow e.g. sudo and steal your password, escalating a local file write into root
That's important only for the people that add relative names (like '.') to their path.
Most people know better.
Otoh, if you don't put your local path first, you can't override system binaries that you want to override.
Also, if something can write into your path, it can probably write to your shell config and/or the environment variables.
by that time it's too late and should have been prevented appearing on the host much earlier
AFAIK it's habit to allow your scripts to override system ones, so you can customize behavior.
I've always seen home dir, homebrew, etc prepending to PATH.
[flagged]
[flagged]
So many headaches could be avoided if we only allowed `[A-Za-z0-9._-]` in paths. (Arguably, even `-` can be problematic.) Encoding issues, expansion, parameter separation, ... and I never saw a convincing case in favor of supporting anything else.
What about people who do not speak English? or even use Latin letters?
The language I grew up with does use non latin letters, I can manage. Then again, it's only four of them, so I get your point ... but I can dream, can't I?
What Latin letter(s) are you referring to?
The (classical) Latin alphabet can be fully described by the English alphabet.
2 replies →
If you're trying to avoid headaches then definitely don't allow -. At least not as the first character in a segment.
And allowing any letters from any language is probably worth the hassle.
I always joked about setting a custom keyboard layout to replace the space char with the underscore char specifically for avoiding spaces in file paths.