Comment by manmal
2 months ago
This tradeoff will be great for self hosted LLMs, because they don’t need large scale batching usually, and less great for cloud providers that do.
2 months ago
This tradeoff will be great for self hosted LLMs, because they don’t need large scale batching usually, and less great for cloud providers that do.
No comments yet
Contribute on Hacker News ↗