Comment by JPLeRouzic
7 hours ago
The author says:
"It used Voronoi Cells and reduced most math to 8- and 16-bit integer calculations with one or two single-precision floating-point calculations."
I wonder if it can be used to speed up LLM inference.
That's what quantization and stuff is, basically. So yes. Been done for years on all neural net types not just LLMs.
Please can you provide some links to intro level articles?