Comment by hnfong
2 years ago
Fabrice Bellard has a project that does precisly this. And does it extremely well, apparently. Previously on HN: http://www.mattmahoney.net/dc/text.html ) . Not sure why it isn't eligible for the Hutter Prize, there's some speculations in the previous discussion but I don't know whether they're true.
Thank you! Turns out that GPT does in fact perform lossless compression if you want it to, like in this demo.
The main issue is that most ML frameworks aren't reliably reproducible, and are not designed for such use cases.
Bellard's solution was to code up his own neural network library in C.
it takes too long to run