Comment by transitorykris
1 year ago
Network availability, latency, privacy, etc. many qualities to consider beyond model size and performance for applications.
1 year ago
Network availability, latency, privacy, etc. many qualities to consider beyond model size and performance for applications.
And cost-efficiency, if I'm using an LLM as an Siri-like assistant on my phone, most of the tasks I'll want it to do won't be that complicated and it would be a waste to send them to some SOTA LLM in the cloud, which I'll have to pay for by a monthly subscription or on a per-token basis.