Comment by Calavar
1 day ago
> {0} initializer in C or C++ for unions no longer guarantees clearing of the whole union (except for static storage duration initialization), it just initializes the first union member to zero. If initialization of the whole union including padding bits is desirable, use {} (valid in C23 or C++) or use -fzero-init-padding-bits=unions option to restore old GCC behavior.
This is going to silently break so much existing code, especially union based type punning in C code. {0} used to guarantee full zeroing and {} did not, and step by step we've flipped the situation to the reverse. The only sensible thing, in terms of not breaking old code, would be to have both {0} and {} zero initialize the whole union.
I'm sure this change was discussed in depth on the mailing list, but it's absolutely mind boggling to me
Fun fact: GCC decided to adopt Clang's (old) behavior at the same time Clang decided to adopt GCC's (old) behavior.
So now you have this matrix of behaviors: * Old GCC: Initializes whole union. * New GCC: Initializes first member only. * Old Clang: Initializes first member only. * New Clang: Initializes whole union.
That's funny and sad at the same time.
And it shows a deeper problem, even though they are willing to align behavior between each other, they failed to communicate and discuss what would be the best approach. That's a bit tragic, IMO
I would argue the even deeper problem is that it's implementation defined. Should be in the spec and they should conform to the spec. That's why I'm so paranoid and zeroize things myself. Too much hassle to remember what is or isn't zero.
1 reply →
Since having multiple compilers is often touted as an advantage, how often do situations like what you're describing happen compared to the opposite — when a second compiler surfaces bugs in one's application or the other compiler?
It is like an era of average.
i will call it "webification" of C!
This was my instinct too, until I got this little tickle in the back of my head that maybe I remembered that Clang was already acting like this, so maybe it won't be so bad. Notice 32-bit wzr vs 64-bit xzr:
Ah, I can confirm what I see elsewhere in the thread, this is no longer true in Clang. That first clang was Apple Clang 17---who knows what version that actually is---and here is Clang 20:
Looks like that change is clang ≤19 to clang 20: https://godbolt.org/z/7zrocxGaq
> This is going to silently break so much existing code
The code was already broken. It was an undefined behavior.
That's a problem with C and it's undefined behavior minefields.
GCC has long been known to define undefined behavior in C unions. In particular, type punning in unions is undefined behavior under the C and C++ standards, but GCC (and Clang) define it.
I have always thought that punning through a union was legal in C but UB in C++, and that punning through incompatible pointer casting was UB in both.
I am basing this entirely on memory and the wikipedia article on type punning. I welcome extremely pedantic feedback.
25 replies →
> type punning in unions is undefined behavior under the C and C++ standards
Union type punning is entirely valid in C, but UB in C++ (one of the surprisingly many subtle but still fundamental differences between C and C++). There's specifically a (somewhat obscure) footnote about this in the C standard, which also has been more clarified in one of the recent C standards.
11 replies →
EDIT: This comment is wrong, see fsmv’s comment below. Leaving for posterity because I’m no coward!
- - -
Undefined behavior only means that the spec leaves a particular situation undefined and that the compiler implementor can do whatever they want. Every compiler defines undefined behavior, whether it’s documented (or easy to qualify, or deterministic) or not.
It is in poor taste that gcc has had widely used, documented behaviors that are changing, especially in a point release.
2 replies →
When you have a big system many people rely on you generally try to look for ways to keep their code working - not look for the changes you’re contractually allowed to make.
GCC probably has a better justification than “we are allowed to”.
> GCC probably has a better justification than “we are allowed to”.
Maybe, but I've seen GCC people justify such changes with little more than "it's UB, we can change it, end of story", so I wouldn't assume it.
Undefined in the standard doesn't mean undefined in GCC. Type-punning through unions has always been a special case that GCC has taken care with beyond the standard.
I thought that {} should always initialize everything regardless of whether or not there is anything in between the braces, and that {0} should only be valid if the first member is a numeric or pointer type (but otherwise has the same effect as {} with nothing in between). I thought that would make more sense, isn't it?
(If you write {} with multiple values when initializing a union, then it should be an error unless all of the values are the same and all of the corresponding members (the first few if you do not explicitly specify which ones) are of the same type as each other.)
C never had {} until C23. In C {0} was the only way to explicitly zero-initialize a structure in a generic manner. It works because in C initializer lists are applied to members as-if nested structures are flattened out lexically.
However, a long time ago C++ went in a completely different direction with initializer lists, and gcc and clang started emitting warnings (in C mode) about otherwise perfectly valid C code, thus the adoption of C++'s {} for C23. {0} is still technically valid C23, though, as well as valid C89, C90, C99, and C11. In fact, reading both C23 and C89 I'm struck by how little the language has changed:
C89 3.5.7p16:
> If the aggregate contains members that are aggregates or unions, or if the first member of a union is an aggregate or union, the rules apply recursively to the subaggregates or contained unions. If the initializer of a subaggregate or contained union begins with a left brace, the initializers enclosed by that brace and its matching right brace initialize the members of the subaggregate or the first member of the contained union. Otherwise, only enough initializers from the list are taken to account for the members of the first subaggregate or the first member of the contained union; any remaining initializers are left to initialize the next member of the aggregate of which the current subaggregate or contained union is a part.
C23 6.7.10p21:
> If the aggregate or union contains elements or members that are aggregates or unions, these rules apply recursively to the subaggregates or contained unions. If the initializer of a subaggregate or contained union begins with a left brace, the initializers enclosed by that brace and its matching right brace initialize the elements or members of the subaggregate or the contained union. Otherwise, only enough initializers from the list are taken to account for the elements or members of the subaggregate or the first member of the contained union; any remaining initializers are left to initialize the next element or member of the aggregate of which the current subaggregate or contained union is a part.
I honestly feel that "uninitialized by default" is strictly a mistake, a relic from the days when C was basically cross-platform assembly language.
Zero-initialized-by-default for everything would be an extremely beneficial tradeoff IMO.
Maybe with a __noinit attribute or somesuch for the few cases where you don't need a variable to be initialized AND the compiler is too stupid to optimize the zero-initialization away on its own.
This would not even break existing code, just lead to a few easily fixed performance regressions, but it would make it significantly harder to introduce undefined and difficult to spot behavior by accident (because very often code assumes zero-initialization and gets it purely by chance, and this is also most likely to happen in the edge cases that might not be covered by tests under memory sanitizer if you even have those).
GCC now supports -ftrivial-auto-var-init=[zero|uninitialized|pattern] for stack variables https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#ind...
For malloc, you could use a custom allocator, or replace all the calls with calloc.
Very nice, did not know about this!
The only problem with vendor extensions like this is that you can't really rely on it, so you're still kinda forced to keep all the (redundant) zero intialization; solving it at the language level is much nicer. Maybe with C2030...
There are many low-level devices where initialization is very expensive. It may mean that you need two passes through memory instead of one, making whatever code you are running twice as slow.
I would argue that these cases are pretty rare, and you could always get nominal performance with the __noinit hint, but I think this would seldomly even be needed.
If you have instances of zero-initialized structs where you set individual fields after the initialization, all modern compiler will elide the dead stores in the the typical cases already anyway, and data of relevant size that is supposed to stay uninitialized for long is rare and a bit of an anti-pattern in my opinion anyway.
Ok, those developers can use a compiler flag. We need defaults that work better for the vast majority.
3 replies →
meh, the compiler can almost always eliminate the spurious default initialization because it can prove that first use is the variable being set by the real initialization. The only time the redundant initialization will be emitted by an optimizing compiler is when it can't prove its redundant.
I think the better reason to not default initialize as a part of the language syntax is that it hides bugs.
If the developers intent is that the correct initial state is 0 they should just explicitly initialize to zero. If they haven't, then they must intend that the correct initial state is the dynamic one in their code and the compiler silently slipping in a 0 in cases the programmer overlooked is a missed opportunity to detect a bug due to the programmer under-specifying the program.
3 replies →
C++26 has everything initialiied by default. The value is not specified though. Implementations are encourage to use something weird to detect using before explict initialization.
Devil's advocate: this would be unacceptable for os kernels and super performance critical code (e.g. hft).
> this would be unacceptable for os kernels
Depends on the boundary. I can give a non-Linux, microkernel example (but that was/is shipped on dozens of millions of devices):
- prior to 11.0, Nintendo 3DS kernel SVC (syscall) implementations did not clear output parameters, leading to extremely trivial leaks. Unprivileged processes could retrieve kernel-mode stack addresses easily and making exploit code much easier to write, example here: https://github.com/TuxSH/universal-otherapp/blob/master/sour...
- Nintendo started clearing all temporary registers on the Switch kernel at some point (iirc x0-x7 and some more); on the 3DS they never did that, and you can leak kernel object addresses quite easily (iirc by reading r2), this made an entire class of use-after-free and arbwrite bugs easier to exploit (call SvcCreateSemaphore 3 times, get sema kernel object address, use one of the now-patched exploit that can cause a double-decref on the KSemaphore, call SvcWaitSynchronization, profit)
more generally:
- unclearead padding in structures + copy to user = infoleak
so one at least ought to be careful where crossing privilege boundaries
No, just throw the __noinit attribute at every place where its needed.
You probably would not even need it in a lot of instances because the compiler would elide lots of dead stores (zeroing) even without hinting.
Would you rather have a HFT trade go correctly and a few nanoseconds slower or a few nanoseconds faster but with some edge case bugs related to variable initialisation ?
You might claim that that you can have both but bugs are more inevitable in the uninitialised by default scenario. I doubt that variable initialisation is the thing that would slow down HFT. I would posit is it things like network latency that would dominate.
3 replies →
It is acceptable enough for Windows, Android and macOS, that have been doing for at least the last five years.
That is the usual fearmongering when security improvements are done to C and C++.
The same OS kernel that zeros out pages before handing them back to me?
1 reply →
Zero initializing often hides real and serious bugs, however. Say you have a function with an internal variable LEN that ought to get set to some dynamic length that internal operations will run over. Changes to the code introduce a path which skips the setting of LEN. Current compilers will (very likely) warn you about the potentially uninitialized use, valgrind will warn you (assuming the case gets triggered), and failing all that the program will potentially crash when some large value ends up in LEN-- alerting you to the issue.
Compare with default zero init: The compiler won't warn you, valgrind won't warn you, and the program won't crash. It will just be silently wrong in many cases (particularly for length/count variables).
Generally the attention to exploit safety can sometimes push us in directions that are bad for program correctness. There are many places where exploit safety is important, but also many cases where its irrelevant. For security it's generally 'safe' is a program erroneously shuts down or does less than it should but that is far from true for software generally.
I prefer this behavior: Use of an uninitialized variable is an error which the compiler will warn about, however, in code where the compiler cannot prove that it is not used the compiler's behavior is implementation defined and can include trapping on use, initializing to zero, or initializing to ~0 (the complement of zero) or other likely to crash pattern. The developer may annotate with _noinit which makes any use UB and avoids the cost of inserting a trap or ~0 initialization. ~0 init will usually fail but seldom in a silent way, so hopefully at least any user reports will be reproducible.
Similar to RESTRICT _noinit is a potential footgun, but its usage would presumably be quite rare and only in carefully maintained performance critical code. Code using _noinit like RESTRICT is at least still more maintainable than assembly.
This approach preserves the compiler's ability to detect programmer error, and lets the implementation pick the preferred way to handle the remaining error. In some contexts it's preferable to trap cleanly or crash reliably (init to ~0 or explicit trap), in others its better to be silently wrong (init 0).
Since C99 lets you declare variables wherever so it is often easy to just declare a variable where it is first set and that's probably best, of course. .. when you can.
Do distros have tooling to deal with this type of change?
I imagine it would be very useful to be able to search through all the C/C++ source files for all the packages in the distro in a semantic manner, so that it understands typedefs and preprocessor macros etc. The search query for this change would be something like "find all union types whose first member is not its largest member, then find all lines of code where that type is initialized with `{0}`".
As a retired Gentoo developer, I can say not really as far as I know. There could be static analysis tools that can find this, but I am not aware of anyone who runs them on the entire distribution.
In theory it's just an extension of IDE tooling. A CLI with a little query language wrapping libclang. In practice I'm sure it's a nightmare just to get 20,000 packages' build systems wrangled such that the right source files get indexed by libclang, and all the endless plumbing for downloading packages and reporting results, and on and on.
1 reply →
Distributions tend to use shell-script-wrapped compilers that can inject additional flags desired by the distribution, and in all likelihood distributions will just add flags that force the old behaviour if there are problems.
> This is going to silently break so much existing code
How much code actually uses unions this way?
> especially union based type punning in C code
I have never done type punning via the GNU C compiler extension in a way that would break because of this. I always assign a value to it and then get out the value from a new type. Do you know of any code that does things differently to be affected by this?
> How much code actually uses unions this way?
I see this change caused Mbed-TLS to start failing its test suite when compiled with GCC 15: https://github.com/Mbed-TLS/mbedtls/issues/9814 (kinda scary since it's a security library). Hopefully other projects with less rigorous test suites aren't using {0} in that way. The Github issue mentions that Clang tried a similar optimization a while ago and backed it out after user complaints, so maybe the same thing will happen with GCC.
GCC’s developers have a strong insistence on standards conformance (minus situations where they explicitly choose to deviate, like type punning in unions) over the status quo. We already went through a much more severe shift with strict aliasing enforcement by GCC and they never changed course. I do not expect this to be any different.
I would guess a lot. People aren't intimately familiar with the standard, and people are lazy when it comes to writing boilerplate like initialization code. And up until now, it just worked, so even a good test suite wouldn't catch it.
EDIT: I initially mentioned type punning for arithmetic, but this compiler change wouldn't affect that
How would that be broken by this? The union will be zero initialized regardless because this change only affects situations where the union members are of different lengths, but for integer to float, the union members should always be the same length or bad things will happen.
2 replies →
lol this is exactly the kind of stuff I expects from C or C++ haha its kinda insane people just decide to do this amidst all the talk about correctness/safety.
I'm skeptical of the claim that this change will "silently break so much existing code". For it to change the behavior of code, the first member would have to be smaller than other members, someone would have to use this construct to initialize union objects, and it would have to affect the behavior. In any case, it's standard for the Fedora, Ubuntu, and Debian developers to go through all the packages and test with new GCC versions before they come out, so that issues are fixed before the new compiler is released.
I have to say, I've read the discussion this generated and it's a bit scary how no one seems to know whether type punning through unions is undefined or not in C, or rather, my conclusion reading it all is more so that many people are wrong and that is defined behavior, but some of the people who are wrong about it are actual GCC compiler developers so it can't be too easy to be right.
Initialisation in C++ is just footguns all the way down.
using UNION was always considered sketchy IMHO. This is trivia for security exploiters?
No. This is how sum types are implemented.
And from a runtime perspective it’s going to be a struct with perhaps more padding. You’ll need more details about your specific threat model to explain why that’s bad.
a quick search says that std::variant is the modern replacement to implement your niche feature "sum types"
9 replies →
There is no reason to use a union unless you're doing some C stuff; in which case just use C.
I feel like once a language is standardized (or reaches 1.0), that's it. You're done. No more changes. You wanna make improvements? Try out some new ideas? Fine, do that in a new language.
I can deal with the footguns if they aren't cheekily mutating over the years. I feel like in C++ especially we barely have the time to come to terms with the unintended consequences of the previous language revision before the next one drops a whole new load of them on us.
> If the size of the new type is larger than the size of the last-written type, the contents of the excess bytes are unspecified (and may be a trap representation). Before C99 TC3 (DR 283) this behavior was undefined, but commonly implemented this way.
https://en.cppreference.com/w/c/language/union
> When initializing a union, the initializer list must have only one member, which initializes the first member of the union unless a designated initializer is used(since C99).
https://en.cppreference.com/w/c/language/struct_initializati...
→ = {0} initializes the first union variant, and bytes outside of that first variant are unspecified. Seems like GCC 15.1 follows the 26 year old standard correctly. (not sure how much has changed from C89 here)
Programming languages are products, that is like saying you want to keep using vi 1.0.
Maybe C should have stop at K&R C from UNIX V6, at least that would have spared the world in having it being adopted outside UNIX.
I liked the idea I heard: internet audiences demand progress, but internet audiences hate change.
If C++ had never been invented, that might have been the case.
5 replies →
I suspect this change was motivated by standards conformance.
The wording of GCC maintainer was "the standard doesn't require it." when they informed Linux kernel mailing list.
https://lore.kernel.org/linux-toolchains/Z0hRrrNU3Q+ro2T7@tu...
1 reply →
> I feel like once a language is standardized (or reaches 1.0), that's it. You're done. No more changes. You wanna make improvements? Try out some new ideas? Fine, do that in a new language.
Thank goodness this is not how the software world works overall. I'm not sure you understand the implications of what you ask for.
> if they aren't cheekily mutating over the years
You're complaining about languages mutating, then mention C++ which has added stuff but maintained backwards compatibility over the course of many standards (aside from a few hiccups like auto_ptr, which was also short lived), with a high aversion to modifying existing stuff.
Perl 6 and Python 3 joined the chat
It's careless development. Why think something in advance when you can fix it later. It works so well for Microsoft, Google and lately Apple. /s
The release cycle of a software speaks a lot about its quality. Move fast, break things has become the new development process.
That does not make sense for anything that exists over decades.
Do you want to be still using Windows NT, or C++ pred 2004 standard or python 2.0
We learn more and need to add to things., Some things we designed 30 years ago were a mistake should we stick with them.
You can't design everything before release for much software. Games you can or bespoke software for a business as you can define what it does, but then the business changes.