It’s by design. The job of autotools is to find “ground truth” about whatever environment you’re compiling against. It’s meant to discover if you can use a feature by actually seeing whether it works, not just by allow-listing a known set of compiler or library versions. This is because the whole point is to allow porting code to any environment where it’ll work, even on compilers you don’t know about. Think back to a time when there were several dozen Unix vendors, and just as many compilers. You don’t want your build script to report it can’t compile something just because it isn’t aware of your particular Unix vendor… you want it to only fail if the thing it’s trying to do actually doesn’t work. The only way to do this is by just testing if certain code compiles and produces the expected result.
Some of the checks are there to tell whether or not the compiler even supports your code. You may not be able to compile your code at all, and the job of the build system is sometimes to just emit useful errors to help the person building the code to understand that they need a compiler which supports [language feature X].
Again, this is intended to be portable software. It is designed to work on lots of OS’s, with lots of compilers, in a lot of future environments that don’t even exist yet.
If you have a security feature for example, which uses the pledge() syscall on OpenBSD, but you can only use that feature on OpenBSD systems, you have two choices:
- Conditionally compile it based on whether you’ve detected that this is an OpenBSD target at build time, or,
- Conditionally compile it based on whether some sample code that uses pledge() builds successfully.
You can’t defer this decision until runtime, because it would require linking to pledge() symbols even though they may not exist on this system, which would cause the executable to fail to link at runtime, unless you completely rearchitected to use a plugin model, which is overkill.
So given the above are your main two options, the latter is preferred mainly because it allows new systems to come in and be compatible with old ones (maybe someone adds pledge() support to Linux one day) without having to fudge the uname command or something. This was super important in the early Unix days… perhaps less so now, but it’s still a good way to write portable software that still can take advantage of platform-specific features.
You'll never make it to runtime if you try to include headers that don't exist. You'll never make it to runtime if you try to link in libraries that don't exist.
It's not by design, unlike what siblings say. It's by accident (so, "legacy", as you put it).
The problem is that already in the 80s there was tons of variability from on Unix system (or version of it) to the next, but there was no standard way of representing what features/standards/APIs/libraries a system supported or had installed. When faced with such a mess people wrote code to detect features are present on the target host.
This then got made into tools with libraries of detection code. Think autoconf/autotools.
Now we also have pkgconfig, but it's too late and it was not enough anyways.
Some things you might only detect at run-time, provided that their ABIs are stable enough that you can copy their headers into your application.
It’s by “design”, in the sense that C and C++ provide no better way to really know for sure that the functions you want to call really exist. In more modern languages we rely on metadata and semver, but none of that exists for C and C++.
Yea, it’s not impossible to do, just not standardized. If you use a library and it provides useful version information, then definitely use it. It’s just that the language or the tooling doesn’t force libraries to have that kind of metadata. Compare that with Rust, where every library you use must come with a standardized manifest that includes a version number, and where they tell library authors up front that they are expected to follow the semver convention.
The fact is that things have good easier over the last 10 or 20 years. It used to be the case that any program targeting Unix had to either spend a lot of time and energy tracking the precise differences between dozens of different commercial Unices, or use autoconf. Autoconf was the project that combined all of that lore into a single place, so that most people didn’t have to know every single detail. But these days most of the Unices are dead and buried, and 99% of all new projects just target Linux. Kinda sucks if you prefer one of the BSDs, or OpenSolaris/Illumos/SmartOS, but it does mean that new Linux developers never have to jump through those hoops and simply never learn about autoconf. And while on the one hand that represents a loss of knowledge and competency for the community (making this type of supply–chain attack much easier), on the other hand autoconf is in practice an abomination. It is (or at least was) extremely useful, but it was implemented in M4 and reading the source code will literally damage your brain.
Back in the day when people compiled source from tarballs on their personal machines, the autoconf script would query your system to see what functionality it supported. It did this by trying to compile small programs for each feature. If the compilation failed it assumes the feature is unavailable on your system and a flag is set/unset for the rest of the build.
Have you ever tried to manually build something from a release tarball by starting with ./configure? If so, have you observed how many times the compiler is invoked in this configure phase before you even run make?
It’s by design. The job of autotools is to find “ground truth” about whatever environment you’re compiling against. It’s meant to discover if you can use a feature by actually seeing whether it works, not just by allow-listing a known set of compiler or library versions. This is because the whole point is to allow porting code to any environment where it’ll work, even on compilers you don’t know about. Think back to a time when there were several dozen Unix vendors, and just as many compilers. You don’t want your build script to report it can’t compile something just because it isn’t aware of your particular Unix vendor… you want it to only fail if the thing it’s trying to do actually doesn’t work. The only way to do this is by just testing if certain code compiles and produces the expected result.
I don’t know. In my code I’d always compile and check at runtime?
Some of the checks are there to tell whether or not the compiler even supports your code. You may not be able to compile your code at all, and the job of the build system is sometimes to just emit useful errors to help the person building the code to understand that they need a compiler which supports [language feature X].
Again, this is intended to be portable software. It is designed to work on lots of OS’s, with lots of compilers, in a lot of future environments that don’t even exist yet.
If you have a security feature for example, which uses the pledge() syscall on OpenBSD, but you can only use that feature on OpenBSD systems, you have two choices:
- Conditionally compile it based on whether you’ve detected that this is an OpenBSD target at build time, or,
- Conditionally compile it based on whether some sample code that uses pledge() builds successfully.
You can’t defer this decision until runtime, because it would require linking to pledge() symbols even though they may not exist on this system, which would cause the executable to fail to link at runtime, unless you completely rearchitected to use a plugin model, which is overkill.
So given the above are your main two options, the latter is preferred mainly because it allows new systems to come in and be compatible with old ones (maybe someone adds pledge() support to Linux one day) without having to fudge the uname command or something. This was super important in the early Unix days… perhaps less so now, but it’s still a good way to write portable software that still can take advantage of platform-specific features.
9 replies →
You'll never make it to runtime if you try to include headers that don't exist. You'll never make it to runtime if you try to link in libraries that don't exist.
Not everything is detectable at runtime, such as syscall numbers (which is what's being tested for here).
2 replies →
It's not by design, unlike what siblings say. It's by accident (so, "legacy", as you put it).
The problem is that already in the 80s there was tons of variability from on Unix system (or version of it) to the next, but there was no standard way of representing what features/standards/APIs/libraries a system supported or had installed. When faced with such a mess people wrote code to detect features are present on the target host.
This then got made into tools with libraries of detection code. Think autoconf/autotools.
Now we also have pkgconfig, but it's too late and it was not enough anyways.
Some things you might only detect at run-time, provided that their ABIs are stable enough that you can copy their headers into your application.
It’s by “design”, in the sense that C and C++ provide no better way to really know for sure that the functions you want to call really exist. In more modern languages we rely on metadata and semver, but none of that exists for C and C++.
Hey, leave C++ out of this. This is a C problem.
https://en.cppreference.com/w/cpp/feature_test
That only works for language features though, it doesn't allow detecting OS/library features.
3 replies →
I never test for this in this way in my C projects. I also rely on version tests only.
Yea, it’s not impossible to do, just not standardized. If you use a library and it provides useful version information, then definitely use it. It’s just that the language or the tooling doesn’t force libraries to have that kind of metadata. Compare that with Rust, where every library you use must come with a standardized manifest that includes a version number, and where they tell library authors up front that they are expected to follow the semver convention.
The fact is that things have good easier over the last 10 or 20 years. It used to be the case that any program targeting Unix had to either spend a lot of time and energy tracking the precise differences between dozens of different commercial Unices, or use autoconf. Autoconf was the project that combined all of that lore into a single place, so that most people didn’t have to know every single detail. But these days most of the Unices are dead and buried, and 99% of all new projects just target Linux. Kinda sucks if you prefer one of the BSDs, or OpenSolaris/Illumos/SmartOS, but it does mean that new Linux developers never have to jump through those hoops and simply never learn about autoconf. And while on the one hand that represents a loss of knowledge and competency for the community (making this type of supply–chain attack much easier), on the other hand autoconf is in practice an abomination. It is (or at least was) extremely useful, but it was implemented in M4 and reading the source code will literally damage your brain.
2 replies →
Back in the day when people compiled source from tarballs on their personal machines, the autoconf script would query your system to see what functionality it supported. It did this by trying to compile small programs for each feature. If the compilation failed it assumes the feature is unavailable on your system and a flag is set/unset for the rest of the build.
Have you ever tried to manually build something from a release tarball by starting with ./configure? If so, have you observed how many times the compiler is invoked in this configure phase before you even run make?