Comment by transitorykris
10 months ago
Network availability, latency, privacy, etc. many qualities to consider beyond model size and performance for applications.
10 months ago
Network availability, latency, privacy, etc. many qualities to consider beyond model size and performance for applications.
And cost-efficiency, if I'm using an LLM as an Siri-like assistant on my phone, most of the tasks I'll want it to do won't be that complicated and it would be a waste to send them to some SOTA LLM in the cloud, which I'll have to pay for by a monthly subscription or on a per-token basis.