Comment by computably
6 days ago
Even ignoring speed per GP, the Hutter Prize's metric includes the size of the decompressor. LLMs would be disqualified for being larger than 1GB.
6 days ago
Even ignoring speed per GP, the Hutter Prize's metric includes the size of the decompressor. LLMs would be disqualified for being larger than 1GB.
Hutter prize does have speed restrictions. If it did not, LLMs would win even with counting the size of the model (which is the most reasonable choice imo) as per the main benchmark: https://www.mattmahoney.net/dc/text.html.