Comment by smeeth

Probably won't have to wait that long. Prism released Bonsai 27B (https://huggingface.co/prism-ml/Ternary-Bonsai-27B-mlx-2bit) as a ternary model a few days ago, its just ~7GB and runs at 44+ t/sec on an m4 max laptop. That's already in the ballpark of active parameter count of most 200B+ models, so we will get a model like this whenever Prism feels like releasing one.

It is debatable if we will actually need that many parameters though, since recursive nets like HRM (https://huggingface.co/sapientinc/HRM-Text-1B) don't need to parametrize as heavily.