Comment by ozgrakkurt
1 day ago
> Consider, for instance, bitcasting a [2]u8 to a u16. Under the old semantics, the result of this operation depends on the target endian: on big-endian targets, the first array element became the 8 most significant bits, whereas on little-endian targets, the first array element became the 8 least significant bits. Under the new semantics, because we only care about logical bit representation (which is endian-agnostic), the operation behaves identically on every target:
This is a huge mistake. You would never expect something like bitCast to do this.
I don't understand this approach. Why change something so simple and low level to be complicated and high level?
Just don't allow casting to u24, as it makes no sense unless you define u24 to be u32 sized as I think c standard does.
I think this approach as an idea is bad but at least just add another built-in that implements this higher level idea to not break a simple expectation and current behavior?
> Just don't allow casting to u24, as it makes no sense unless you define u24 to be u32 sized as I think c standard does.
The reason u32->u24 casting must be well defined is because some hardware (e.g. many GPUs, microcontrollers) only have floating point multipliers. A 24 bit unsigned integer (stored in a 32 bit register) can be losslessly converted to a 32 bit float by the hardware, multiplied, then converted back.
This is much faster than doing 32 bit multiplication in software, however, you still need to tell the compiler about this constraint.
I am criticizing the part where they allowed [3]u8 to u24 bitCast in the first place. It doesn't make sense logically as u24 is likely not 24 bits in any targets let alone portably on every target.
Interpreting u24 like it is actually 24 bits sounds like programming in crazy land since it is not 24 bits in any relevant architecture afaik.
They didn't allow []u24 with a similar rationale as far as I can remember. I agree with this as someone programming at this level should be able to understand there is no real u24 layout and they should use []u32. Going with the same magical rational they went with here, compiler should generate unaligned u24 loading code when you use []u24 since it is "logically 24 bits"
The ease of dealing with arbitrary bit-width integers and packed structs is actually one of the 'killer features' for me in zig.
Zig natively supports arbitrary bit-width integers, the ABI is defined and you could simply think it as a slice of the next larger backing integer.
The[3]u8 to u24 bitCast will simply be backed by a 32bit int, using the same ABI. As you have u1 - u65535, sometimes it can be multiple words.
The 24 Bits (3 Bytes) [3]u8 to u24 example is exactly related to utf-8 that covers all the languages but excludes the emojis.
There are very valid use cases when you want to limit utf-8 to U+0000-U+FFFF, and it is valuable if your language allows you to make those decisions.
Remember, in zig packed structs are just integers and integers are just a group of logically consecutive bits.
Arrays like []u24 do not have the same ABI, arrays are not bit/byte packed, are not universally LSB across archs etc..
The compiler isn't producing unaligned code, don't confuse the abstraction with the concrete implementation. And yes [8]u1 and [8]u8 are exactly the same size and shape, even though they are arrays.
My current project is parsing ELF/Macho files, I can easily have zero allocations in my hot path with zig, the same is far more challenging in C, so I am biased, especially with zig allowing methods on structs.
And yes, I do use that crazy casting to 0xdeadbeef and other ascii metadata that is in those files.
To be clear here, I am not trying to prove you wrong, this is one of the places zig is very different and (IMHO) useful. Especially with streaming data or where you have network ordering etc... It is so nice to only cast what you need to but it does take a little while to wrap your head around how this interacts with buffers which are not your native endianness. At least for me, once I figured out to separate the shape of those data streams from their values it was super useful.
9 replies →
> many GPUs
Citation please - every single GPU in the literal world supports integer arithmetic for operating on tid, gid, etc.
From page 175 of the AMD CDNA4 ISA:
https://www.amd.com/content/dam/amd/en/documents/instinct-te...
> V_MUL_U32_U24
>,Multiply two unsigned 24-bit integer inputs and store the result as an unsigned 32-bit integer into a vector register. D0.u32 = 32'U(S0.u24) * 32'U(S1.u24)
> Notes
> This opcode is expected to be as efficient as basic single-precision opcodes since it utilizes the single-precision floating point multiplier. See also V_MUL_HI_U32_U24.
Nvidia GPUs used to do the same thing and theres a umul24 intrinsic if you care to use it.
https://stackoverflow.com/questions/5544355/cuda-umul24-func...
This is super-super-niche since it basically only applies to 32-bit integer multiplication.
You likely won't run into it unless you're doing high performance embedded systems or GPU programming on non-NVDIA cards, and for some unknowable reason, your workload does a 32-bit integer multiplication in the hot path.
2 replies →
While the GP might be technically wrong in a narrow sense, GPUs are built for FP, and that's what you want to be doing if you're using them as accelerators.
3 replies →
GCC has had __int24 for the AVR backend for some time. Useful for larger integers than int16_t while saving 25% over a 32-bit value. C23 does not mandate padding for _BitInt types. It is wrong to assume that will happen or is the optimal implementation for portable code.
Thanks for the context, but what I am criticising is this part:
> it became allowed to use @bitCast to reinterpret a [3]u8 as a u24
This cant't make sense unless u24 is defined to be 24bits in the first place. It is just silly to allow something like this. It would make so much more sense to me if they started disallowing this or just even print a deprecation notice for it for one release version.
> Useful for larger integers than int16_t while saving 25% over a 32-bit value
You can't even do []u24 in zig as far as I can remember and understand anyway so this is only happening in a packed struct context.
C doesn't mandate padding but C compilers allow having pointers and arrays of irregular _BitInt types as far as I can understand.
In this [1] document, in Abi considerations section, it writes that it is defined to have next-power-of-two layout size.
Also here (for RISCV) [2] it seems like it is defined with next-power-of-two layout.
Also the document here (for x86_64) defines it similarly [3]
[1] https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2709.pdf
[2] https://github.com/riscv-non-isa/riscv-elf-psabi-doc/issues/...
[3] https://gitlab.com/x86-psABIs/x86-64-ABI/-/tree/master?ref_t...
> This cant't make sense unless u24 is defined to be 24bits in the first place
It's worth remembering that zig is a ~hll that should be platform agnostic. suppose someone built a byte-chip with a 24 bit word. the "new" zig way of doing things will be more portable and slot right in, and support 32- and 16-bit datatypes just fine.
I sort of agree ... bit casting from an N width integer into an array of ... woah ... that's too far. It's bitcast not byte-cast which has an implied reinterpretation on a same or smaller word size in the cpu.
Once you see that the fact somebody has a u24 in their code is between them and the compiler alone.
As others probably noted byte casting (keeping the same endianess) is what unions are for.
> This is a huge mistake. You would never expect something like bitCast to do this.
Is there at least some sort of @transmute or something ? If Zig wants to say "bitCast" means this odd operation, but provides the thing most people actually want under some plausible name that's just an extra thing to learn which seems OK.
@intCast
So, since I don't write Zig I had to go look this up, to save anyone else the bother this is what Rust would call an 'as' cast or C programmers might think of as a value cast, it's going to try to make a value which has a similar meaning but of another type, which may be arbitrarily expensive. What people often want here is a transmute, Rust's core::mem::transmute which changes nothing about the bits except what those bits mean, since the bits didn't change and the machine only has bits anyway this is "free".
2 replies →
I may be damaged from working on IC hardware design and various weird architectures, but I truly can’t comprehend why you’d think this doesn’t make sense.
Yeah, if your architecture doesn’t support 24-bit int it maps to 32-bits. But it also declares that the numbers you’re storing should never be larger than 2^24. It’s about type safety, and also run time checks in safe mode I believe. Bitcasting three bytes to a 24-bit type makes just as much sanse as casting 4 bytes to 32-bit. Theres zero reasons to introduce arbitrary artificial constraints on what you can do based on details of (most of) the underlying architectures, which doesn’t even matter for the operation you’re performing.
If the architecture supports 3 byte types that means it needs to support 3 byte alignments and their powers 9, 27, 81, etc. The easiest way to support this is to always map every 3-byte read operation to two 2-byte reads and then use multiplexers to recombine it into a 24 bit data type.
Of course you could also go crazy and store data in 24 bit blocks in your SRAM. That kind of ruins the 8 bit and 16 bit reads though.
If I understand it correctly, it basically boils down to copying bits from the source to the destination, in order from the least significant bit to the most significant bit. It's not equivalent to C++'s reinterpret_cast.
I'm no Zig expert, but if you want endian-dependent semantics I'd assume either @ptrCast or a packed union would do the job.
But doesn't that show why this is a bad idea? If I understand correctly, this code:
...will now succeed or fail depending on the endianness of the target. That looks like the type of footgun that will bring decades of joy.
zig does not allow arrays in packed structs/unions specifically for endianness reasons (there may be other reasons as well but endianness is what i know of)
2 replies →
I wonder if packed union also got/will get the same "logical bits" treatment?
1 reply →
You don't need to use @bitCast for the behavior you're talking about. @ptrCast still exists.
@ptrCast,
> Converts a pointer of one type to a pointer of another type. [1]
[1] https://ziglang.org/documentation/master/#toc-ptrCast
So it is not the same.
You could use it to define a function that implements bitCast. Which defeats the purpose of having any @bitCast intrinsic instead of using @mempcy for everything
Take the address and deref afterwards, and it's exactly the same. Or to say another way: if you want bits to be reinterpreted raw as if they're in memory, then... put them in memory, then reinterpret them.
> You could use it to define a function that implements bitCast. Which defeats the purpose of having any @bitCast intrinsic
Yes, and this is one reason @bitCast was changed to have different semantics that are not trivially achieved with @ptrCast.
3 replies →
I understand the reaction, but I don't agree. I suggest reading the associated proposal[0] along with the devlog, and having a real think about what's going on here. I'm responding to you saying that you "don't understand" the approach: reasonable, and resembles my initial reaction.
I was inclined to agree with you, but what decided it for me is that Zig has another mechanism for "reinterpret bytes". It's exposed on the stdlib as std.mem.asBytes, but this is literally a wrapper for the following:
So nothing is lost here: if you need, for whatever reason (and those do exist), to get a raw array of underlying bytes, you absolutely may. Std.mem also has bytesToValue(T, bytes) T, which makes a copy. All the ingredients are there, and this family of mem functions are thin wrappers over builtins, which boil down to pointer casting, dereferencing, and comptime magic.
Also worth noting: packed structs in Zig are already defined as logically little-endian: the first field is of low significance, the second is above that, and so on. So this makes `@bitCast` consistent with an existing convention of treating integers as logically little-ended, without regard to how they're actually arrayed in memory.
Plus it stands to make low-level bit-twiddling, using oddly-sized integers, optimize better. I like that, especially when what we trade for that is: nothing. Nothing at all, this is a pure win.
I'd even guess it's that rare language update which silently fixes buggy code, where someone figured "well, basically everything is little-endian already" (or just didn't think about it), and now that code works properly on big-endian machines.
[0]: https://github.com/ziglang/zig/issues/19755
To me it makes sense. If you don't know what endianness is, it doesn't make sense that a program you write in one programming language works for one target but doesn't work for the other.
I think endianness is the footgun that Zig is solving, rather than Zig being the one introducing a footgun when you deal with endianness.
> If you don't know what endianness is
It is not feasible for someone to write endian portable code in a language like Zig without understanding what endianness is imo. Regardless of how they change @bitCast there will be other cases that break this like doing @ptrCast + @memcpy.
Also this breaks currently written code that is endian portable and uses @byteSwap like it is done in most other programming languages that do these things.
I completely disagree.
> As a general rule, the new semantics tend to match the behavior of the old semantics on little-endian targets.
They've basically said that bit casting is going to be little endian. This simplifies things for the 100% of people that are on little endian machines, while making the code still work for the 0% of people (rounded to the nearest 0.0000001%) that are using big endian machines.