Comment by Arnavion
2 years ago
Tangentially related, there's another "unfortunate" detail of Rust that makes some structs bigger than you want them to be. Imagine a struct Foo that contains eight `Option<u8>` fields, ie each field is either `None` or `Some(u8)`. In C, you could represent this as a struct with eight 1-bit `bool`s and eight `uint8_t`s, for a total size of 9 bytes. In Rust however, the struct will be 16 bytes, ie eight sequences of 1-byte discriminant followed by a `uint8_t`.
Why? The reason is that structs must be able to present borrows of their fields, so given a `&Foo` the compiler must allow the construction of a `&Foo::some_field`, which in this case is an `&Option<u8>`. This `&Option<u8>` must obviously look identical to any other `&Option<u8>` in the program. Thus the underlying `Option<u8>` is forced to have the same layout as any other `Option<u8>` in the program, ie its own personal discriminant bit rounded up to a byte followed by its `u8`. The struct pays this price even if the program never actually constructs a `&Foo::some_field`.
This becomes even worse if you consider Options of larger types, like a struct with eight `Option<u16>` fields. Then each personal discriminant will be rounded up to two bytes, for a total size of 32 bytes with a quarter (or almost half, if you include the unused bits of the discriminants) being wasted interstitial padding. The C equivalent would only be 18 bytes. With `Option<u64>`, the Rust struct would be 128 bytes while the C struct would be 72 bytes.
You *can* implement the C equivalent manually of course, with a `u8` for the packed discriminants and eight `MaybeUninit<T>`s, and functions that map from `&Foo` to `Option<&T>`, `&mut Foo` to `Option<&mut T>`, etc, but not to `&Option<T>` or `&mut Option<T>`.
https://play.rust-lang.org/?version=stable&mode=debug&editio...
You have to implement the C version manually, so it's not that odd you'd need to do the same for Rust?
You've described, basically, a custom type that is 8 Options<u8>s. If you start caring about performance you'll need to roll your own internal Option handling.
>You have to implement the C version manually
There's no "manually" about it. There's only one way to implement it in C, ie eight booleans and eight uint8_ts as I described. Going from there to the further optimization of adding a `:1` to every `bool` field is a simple optimization. Reimplementing `Option` and the bitpacking of the discriminants is much more effort compared to the baseline implementation of using `Option`.
The alternative is `std::optional` which works exactly the same as Rust's `Option` (without the niche optimisation).
I'm not a C programmer but I imagine you could make something like `std::optional` in C using structs and macros and whatnot.
But it's not any more work than it would take in C. What does it matter how much work it is relative to rust's happy path?
> You can implement the C equivalent manually of course
But you have to implement the C version manually as well.
It's not really a downside to Rust if it provides a convenient feature that you can choose to use if it fits your goals.
The use case you're describing is relatively rare. If it's an actual performance bottleneck then spending a little extra time to implement it in Rust doesn't seem like a big deal. I have a hard time considering this an "unfortunate detail" to Rust when the presence of the Option<_> type provides so much benefit in typical use cases.
I answered this in the other subthread already.