I've read that a handful of times (whenever trying to figure out where to put some new random thing), and still have never come to a clear conclusion. Even better, because there are so many similar places, you might choose completely different ones depending on the day of the week and your current mood.
Too much choice for things like this is harmful IMO. Deep down I truly couldn't care less where the files end up, as long as that place is the 'right' place. There are too many 'right' places which makes it hard to find random things at a later date or when on a box you're not super familiar with. It's also a complete waste of time to think about it at all.
It’s not just you: Every distro is its own special snowflake and patches the programs they distribute to store files in a different place.
The “standard” doesn’t tell you what directory structure to use inside /etc to group related config files. The “standard” doesn’t tell you where an HTTP server should serve its files. Everyone just does their own thing which makes upstream docs incorrect and useless for newcomers.
> The “standard” doesn’t tell you what directory structure to use inside /etc to group related config files. The “standard” doesn’t tell you where an HTTP server should serve its files. Everyone just does their own thing which makes upstream docs incorrect and useless for newcomers.
The FHS, does actually answer both of of those questions. Files inside /etc/ should be grouped in subdirectories[0] andd the HTTP server should serve user-specified website files from /srv[1] and normal distro-provided files (such as the apache test page) from /var[2].
> I've read that a handful of times
(whenever trying to figure out where to
put some new random thing), and still have
never come to a clear conclusion.
So, given some data, say a file and/or
directory, maybe from saving a Web page,
that is relevant to subjects A, K, T, and
Z, where in the file system directory
trees to put that data?
My solution: Put the data in a directory
for one of the subjects A, K, T, or Z
without thinking very hard about which of
these. Then go to a file I call FACTS.DAT
(right, an old idea with an old 8.3 file
name!). I maintain that file with a few,
simple editor macros. So, sure, the file
is a catch-all for entries of random short
facts. And each entry starts with a
time-date stamp and a list of key words.
So, in the case of subjects A, K, T, or Z,
include the key words appropriate for each
of those. Then in the body of the
entry, put the tree name of the
file/directory where did store the data.
In a few seconds with my favorite text
editor I can append an entry or search for
an entry.
So far this year I have put 686 entries in
the file FACTS.DAT for about 2.1 entries
per day. For anything like current
personal computers, handling such a file
is trivial.
Yeah, except that tells me nothing useful... The question is exactly the same: So where do I install this random binary I downloaded from the internet or compiled myself? Is it /opt, /usr/bin, /usr/local/bin, or /bin? Where do I put the dependencies I compiled for this software - /usr/lib, /usr/local/lib, /lib, /opt/lib, /opt/<app name>/lib, or what?
To add: when you install software yourself you choose this, when your install software from e.g. a distribution package it is chosen by the package maintainers, and to a larger extent the maintainers of the distribution.
This is one of the big advantages of using a pre-made advantages of using a ready-made Linux distribution: beyond the convenience of having an installer or easy to install packages, you get some assurance that the system as a whole has been thoughtfully put together.
Arch Linux for example symlinks /bin and /sbin to /usr/bin and /lib to /usr/lib among other things.
Is your account the only account that's expected to run the binary? If so, then `$HOME/bin` is a perfectly acceptable (albeit not standard) place to put it.
If you expect other users to be able to execute the program, then you should put it in either `/usr/bin` or `/usr/local/bin`, depending on whether the former is already being used by a package manager. `/opt` is generally for self-contained software that doesn't play nicely with the rest of the system, but might still be installable through the default package manager.
I started out using $HOME/bin, but a fair amount of stuff assumes a /usr- or /usr/local-style folder structure when doing make install, so I've settled on using $HOME/usr/bin instead, so that programs can create $HOME/usr/include and $HOME/usr/share and whatever, without trampling on stuff in my home folder.
Can't remember the last time I had a problem arranging this. If using autotools, which covers 95+% of stuff, it's usually a question of something like "./configure --prefix=$HOME/usr".
(If I want to share stuff between users, /usr/local/ is of course a better place. macOS is a bit more restrictive, so I have a separate user for this, whose /usr folder is readable by everybody.)
Follow your distribution. For example Arch Linux provides PKGBUILDs for official repos and AUR. Most of the time someone has already published PKGBUILD, but if not I just patch accordingly.
And conditions that formed separation are long gone, Arch Linux symlinks most of it:
The standard is, indeed, excessively vague because it was written to let many existing implementations be conformant as is, though I’d say it’s still more helpful than many other standards with that deficiency. There’s a method to it, however:
- Things installed in /, if it’s different from /usr, are generally not to be touched;
- Things installed in /usr are under the distro’s purview or otherwise under a package manager, any modifications are on pain of confusing it;
- Things installed in /usr/local are under the admin’s purview and unmanaged one-offs, there are always some but overuse will lead to anarchy;
- Things installed in /opt are for whatever is so foreign and hopeless in not conforming to the usual factoring that you just give up and put it in its own little padded cell (hello, Mathematica);
- Everything is generally configured using files in /etc, possibly with the exception of some of the special snowflakes in /opt; the package manager will put config files meant to be edited there and expect the admin to merge any changes in manually, and sometimes put default settings meant to be overridden by them in /usr/share (see below)—both approaches can be problematic, but the difficulty is with migrating configuration in general, not the FHS as such.
There used to be additional hierarchies like /usr/X11R6, and even a /usr/etc on some (non-Linux?) systems, but AFAIU everyone agrees their existence makes no sense (anymore?), so much that even FHS doesn’t lower itself to permitting them.
The distinction between / and /usr might appear to be pointless as well, and nowadays it might be (some distros symlink them together), but previously (especially before initial ramdisks were widespread) stuff in / was whatever was needed to bring up the system enough that it could netmount a shared /usr.
Inside each of /, /usr and /usr/local there is bin for things that are supposed to be directly executable, whether binary or a script and all in a single place; share and lib for other portable and non-portable (usually but not necessarily text and binary) shared files, respectively, segregated by application or purpose; finally, due to the dominance of C ABIs and APIs on Unices, the top level of lib also hosts C and C++ library files and there’s an additional directory called include for the headers required to use them. Some people also felt that putting auxiliary executables (things like cc1, the first pass of the C compiler) inside lib was awkward so they created libexec for that purpose, but I don’t think the distinction turned out to be particularly useful so not all distros maintain it.
That’s it, basically. There are subtler but logical points (files vs subdiretories in /etc) and things people haven’t found an obviously superior solution for (multilib and cross environments), and I made no attempt to be historically accurate (the original separation of / and /usr happened for intensely silly reasons), but those are the fundamental principles of the system, and I feel it does make sense as a coherent implementation of a particular design. Other designs are possible (separation by application or package not purpose, Plan 9-ish overlays, NixOS’s isolated environments), but that’s a discussion on a different level; the point is that this one is at the very least internally consistent.
Re the unfriendly names ... I honestly don’t know. Newbie-friendliness matters, but it’s not the only thing that does; particularly in a system intended for interactive text-mode use, concise names have a quality of their own. There’s a reason I’m more willing to reach for curl and jq rather than for httpx and lxml, for regular expressions rather than for Parsec, and even for cmd.exe, as miserable as it is, rather than for PowerShell.
I feel weird that no HCI people seem to have seriously considered the tension between interactive and programmatic environments and what the text-mode user’s experience in Unix says about it, but even Tcl, which is in many ways a Bourne shell done right, loses something in casual REPL use when it eliminates (as far as idiomatic libraries are concerned) short switches. Coming up with things like rsync -avz or objdump -Ctsr is not very pleasant initially, but I certainly wouldn’t want to type out the longhand form that would be the only possible one in most programming languages (even if I find their syntax beautiful, e.g. Smalltalk/Self).
>the original separation of / and /usr happened for intensely silly reasons
As I recall, there were very good reasons for separating / and /usr (as well as /home and /var). The biggest one was that various Unix kernels would panic[0] if / was full. But that issue was almost universally fixed by 1990 or so.
And netmounts of pretty much everything other than / were pretty common for many years, due to the high cost of storage.
So no, the reasons weren't silly, they just don't apply to more modern systems.
I feel like it just highlights the problem of how antiquated and confusing linux terminology that so many of those reference "single-user mode", used to refer to booting into root, when the vast majority of computing devices a given user will interact with only have a single actual user, making this a confusing and almost meaningless distinction to someone not already intimate familiar with linux.
I've read that a handful of times (whenever trying to figure out where to put some new random thing), and still have never come to a clear conclusion. Even better, because there are so many similar places, you might choose completely different ones depending on the day of the week and your current mood.
Too much choice for things like this is harmful IMO. Deep down I truly couldn't care less where the files end up, as long as that place is the 'right' place. There are too many 'right' places which makes it hard to find random things at a later date or when on a box you're not super familiar with. It's also a complete waste of time to think about it at all.
It’s not just you: Every distro is its own special snowflake and patches the programs they distribute to store files in a different place.
The “standard” doesn’t tell you what directory structure to use inside /etc to group related config files. The “standard” doesn’t tell you where an HTTP server should serve its files. Everyone just does their own thing which makes upstream docs incorrect and useless for newcomers.
> The “standard” doesn’t tell you what directory structure to use inside /etc to group related config files. The “standard” doesn’t tell you where an HTTP server should serve its files. Everyone just does their own thing which makes upstream docs incorrect and useless for newcomers.
The FHS, does actually answer both of of those questions. Files inside /etc/ should be grouped in subdirectories[0] andd the HTTP server should serve user-specified website files from /srv[1] and normal distro-provided files (such as the apache test page) from /var[2].
[0]: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s07.htm...
[1]: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s17.htm...
[2]: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05.html#p...
2 replies →
> I've read that a handful of times (whenever trying to figure out where to put some new random thing), and still have never come to a clear conclusion.
So, given some data, say a file and/or directory, maybe from saving a Web page, that is relevant to subjects A, K, T, and Z, where in the file system directory trees to put that data?
My solution: Put the data in a directory for one of the subjects A, K, T, or Z without thinking very hard about which of these. Then go to a file I call FACTS.DAT (right, an old idea with an old 8.3 file name!). I maintain that file with a few, simple editor macros. So, sure, the file is a catch-all for entries of random short facts. And each entry starts with a time-date stamp and a list of key words. So, in the case of subjects A, K, T, or Z, include the key words appropriate for each of those. Then in the body of the entry, put the tree name of the file/directory where did store the data.
In a few seconds with my favorite text editor I can append an entry or search for an entry.
So far this year I have put 686 entries in the file FACTS.DAT for about 2.1 entries per day. For anything like current personal computers, handling such a file is trivial.
The idea works great!
Yeah, except that tells me nothing useful... The question is exactly the same: So where do I install this random binary I downloaded from the internet or compiled myself? Is it /opt, /usr/bin, /usr/local/bin, or /bin? Where do I put the dependencies I compiled for this software - /usr/lib, /usr/local/lib, /lib, /opt/lib, /opt/<app name>/lib, or what?
I was taught /usr/local/bin
/opt is for standalone packages, so if it’s a single file, no.
/bin is only for stuff needed on single user mode, so probably not (unless that’s what the binary is for.
/usr/bin is going to typically contain files installed by your package manager and should probably be left unaltered by human hands.
The deps I would assume /usr/local/lib but it hasn’t ever come up for me.
Fun fact: Debian is working towards[1] and Arch already has merged / and /usr. /bin is a soft link to /usr/bin and similarly with sbin and lib.
[1]: https://wiki.debian.org/UsrMerge
To add: when you install software yourself you choose this, when your install software from e.g. a distribution package it is chosen by the package maintainers, and to a larger extent the maintainers of the distribution.
This is one of the big advantages of using a pre-made advantages of using a ready-made Linux distribution: beyond the convenience of having an installer or easy to install packages, you get some assurance that the system as a whole has been thoughtfully put together.
Arch Linux for example symlinks /bin and /sbin to /usr/bin and /lib to /usr/lib among other things.
Is your account the only account that's expected to run the binary? If so, then `$HOME/bin` is a perfectly acceptable (albeit not standard) place to put it.
If you expect other users to be able to execute the program, then you should put it in either `/usr/bin` or `/usr/local/bin`, depending on whether the former is already being used by a package manager. `/opt` is generally for self-contained software that doesn't play nicely with the rest of the system, but might still be installable through the default package manager.
$HOME/.local is the equivalent if /usr/local for per-user stuff.
1 reply →
I started out using $HOME/bin, but a fair amount of stuff assumes a /usr- or /usr/local-style folder structure when doing make install, so I've settled on using $HOME/usr/bin instead, so that programs can create $HOME/usr/include and $HOME/usr/share and whatever, without trampling on stuff in my home folder.
Can't remember the last time I had a problem arranging this. If using autotools, which covers 95+% of stuff, it's usually a question of something like "./configure --prefix=$HOME/usr".
(If I want to share stuff between users, /usr/local/ is of course a better place. macOS is a bit more restrictive, so I have a separate user for this, whose /usr folder is readable by everybody.)
2 replies →
Follow your distribution. For example Arch Linux provides PKGBUILDs for official repos and AUR. Most of the time someone has already published PKGBUILD, but if not I just patch accordingly.
And conditions that formed separation are long gone, Arch Linux symlinks most of it:
The standard is, indeed, excessively vague because it was written to let many existing implementations be conformant as is, though I’d say it’s still more helpful than many other standards with that deficiency. There’s a method to it, however:
- Things installed in /, if it’s different from /usr, are generally not to be touched;
- Things installed in /usr are under the distro’s purview or otherwise under a package manager, any modifications are on pain of confusing it;
- Things installed in /usr/local are under the admin’s purview and unmanaged one-offs, there are always some but overuse will lead to anarchy;
- Things installed in /opt are for whatever is so foreign and hopeless in not conforming to the usual factoring that you just give up and put it in its own little padded cell (hello, Mathematica);
- Everything is generally configured using files in /etc, possibly with the exception of some of the special snowflakes in /opt; the package manager will put config files meant to be edited there and expect the admin to merge any changes in manually, and sometimes put default settings meant to be overridden by them in /usr/share (see below)—both approaches can be problematic, but the difficulty is with migrating configuration in general, not the FHS as such.
There used to be additional hierarchies like /usr/X11R6, and even a /usr/etc on some (non-Linux?) systems, but AFAIU everyone agrees their existence makes no sense (anymore?), so much that even FHS doesn’t lower itself to permitting them.
The distinction between / and /usr might appear to be pointless as well, and nowadays it might be (some distros symlink them together), but previously (especially before initial ramdisks were widespread) stuff in / was whatever was needed to bring up the system enough that it could netmount a shared /usr.
Inside each of /, /usr and /usr/local there is bin for things that are supposed to be directly executable, whether binary or a script and all in a single place; share and lib for other portable and non-portable (usually but not necessarily text and binary) shared files, respectively, segregated by application or purpose; finally, due to the dominance of C ABIs and APIs on Unices, the top level of lib also hosts C and C++ library files and there’s an additional directory called include for the headers required to use them. Some people also felt that putting auxiliary executables (things like cc1, the first pass of the C compiler) inside lib was awkward so they created libexec for that purpose, but I don’t think the distinction turned out to be particularly useful so not all distros maintain it.
That’s it, basically. There are subtler but logical points (files vs subdiretories in /etc) and things people haven’t found an obviously superior solution for (multilib and cross environments), and I made no attempt to be historically accurate (the original separation of / and /usr happened for intensely silly reasons), but those are the fundamental principles of the system, and I feel it does make sense as a coherent implementation of a particular design. Other designs are possible (separation by application or package not purpose, Plan 9-ish overlays, NixOS’s isolated environments), but that’s a discussion on a different level; the point is that this one is at the very least internally consistent.
Re the unfriendly names ... I honestly don’t know. Newbie-friendliness matters, but it’s not the only thing that does; particularly in a system intended for interactive text-mode use, concise names have a quality of their own. There’s a reason I’m more willing to reach for curl and jq rather than for httpx and lxml, for regular expressions rather than for Parsec, and even for cmd.exe, as miserable as it is, rather than for PowerShell.
I feel weird that no HCI people seem to have seriously considered the tension between interactive and programmatic environments and what the text-mode user’s experience in Unix says about it, but even Tcl, which is in many ways a Bourne shell done right, loses something in casual REPL use when it eliminates (as far as idiomatic libraries are concerned) short switches. Coming up with things like rsync -avz or objdump -Ctsr is not very pleasant initially, but I certainly wouldn’t want to type out the longhand form that would be the only possible one in most programming languages (even if I find their syntax beautiful, e.g. Smalltalk/Self).
>the original separation of / and /usr happened for intensely silly reasons
As I recall, there were very good reasons for separating / and /usr (as well as /home and /var). The biggest one was that various Unix kernels would panic[0] if / was full. But that issue was almost universally fixed by 1990 or so.
And netmounts of pretty much everything other than / were pretty common for many years, due to the high cost of storage.
So no, the reasons weren't silly, they just don't apply to more modern systems.
[0] https://en.wikipedia.org/wiki/Kernel_panic
2 replies →
Thank you for the thoughtful reply, the point about netmounting shared usr makes it much easier to understand.
Where?
See my post to this thread at
https://news.ycombinator.com/item?id=29198222
> So where do I install this random binary I downloaded from the internet or compiled myself?
In your home directory.
Wherever you want. All of the above, or none. It really is up to you.
That's exactly the problem. This leads to mess. The Windows model of C:\Program Files\<app name> is much better.
10 replies →
I feel like it just highlights the problem of how antiquated and confusing linux terminology that so many of those reference "single-user mode", used to refer to booting into root, when the vast majority of computing devices a given user will interact with only have a single actual user, making this a confusing and almost meaningless distinction to someone not already intimate familiar with linux.