Comment by OtherShrezzing
7 hours ago
I don’t think anyone needs to compete with the LLM SOTA to get the benefits of these technologies on-device.
Consumers don’t need a 100k context window oracle that knows everything about both T-Cells and the ancient Welsh Royal lineage. We need focused & small models which are specialised, and then we need a good query router.
We need them for what? Specialized models seem to provide a value comparable to what we've been doing with machine learning for eons, just more inefficient to train and to run.