Comment by ExxKA
7 days ago
From an investors perspective, this is truly a paradigm shift - this will kill a whole range of startups in Europe which were packaging privacy and wrapping around large hosted models. There's absolutely no reason to use a "Privacy GPT tm" provider, then I have it all on my own laptop - There is also no need for banks or other regulated institutions to rely on those providers when they can selfhost with this much intelligence on tap.
It will when they can get the performance up a bit.
My brief experiments with the ternary version suggest that it broadly meets their claim to be a 27B model that fits in much less RAM, that is for sure. It is about as fast as the underlying Qwen 27B but it gets stuck in reasoning loops quite easily.