Comment by michaelt
3 months ago
No need to unpack for inference. As things like CUDA kernels are fully programmable, you can code them to work with 4 bit integers, no problems at all.
3 months ago
No need to unpack for inference. As things like CUDA kernels are fully programmable, you can code them to work with 4 bit integers, no problems at all.
No comments yet
Contribute on Hacker News ↗