Comment by catgary
2 years ago
Yeah, it’s not (entirely) the students’ faults that this slipped through peer review. I don’t envy the whiplash they’re going to experience over the next few weeks.
If I was the graduate chair of their department I might schedule a meeting with their supervisor to sort out how this happened.
What about the difference in CPU cost, RAM cost, and GPU training hours, though? What about the comparative Big-E's of the models?
Great topic: Model minification and algorithmic omplexity
These are good research topics, but then you really need to be comparing to other models in the same class.
The only other super cheap model they compare with is FastText, and FastText beat them quite substantially.
Gzip fails at audio and video compression IIRC?
Are time-domain signals better compressed with A/V codecs (compression-decompression); and does this gzip finding suggest that other compression algorithms likely to outperform LLMs at certain tasks like computational complexity, at least?