← Back to context

Comment by PaulRobinson

6 hours ago

Local LLMs are the future, and one of the reasons I think the data centre furore is just going to end in a market crash.

LLMs that can reliably be used for control problems are the future, and I think classic/deep RL has generally been overlooked for years for a whole host of problems by wider industry because it felt inaccessible. The first thing I thought of when I saw Jev (and then Laya), was "this might move the needle in a really, really interesting way".

Local LLMs that can reliably be used for control problems smash through a lot of barriers I'm interested in, and this intrigues me a lot. Guess I'm about to become a big Laya fan if it can run on this kind of hardware to this performance.

6 comments

PaulRobinson

Reply
ipsi  4 hours ago

The future for whom? The general public? Not a chance, no way, not unless it's able to run on a phone (anywhere from 20-40% of internet users, world-wide, are phone-only).

For companies? I think that's a lot more plausible, as that's mostly just a question of money - is it cheaper to run and administrate our own models, or outsource that?

For technically inclined users? I think that's unlikely unless they're able to operate on relatively cheap hardware while still being just as good as the hosted models. And by that I don't mean "a mac studio," that's far more money than I think is reasonable. A single RTX 5080, maybe, once memory prices start to drop.

  • Izmaki  3 hours ago

    Compare the games your average high-end smartphone can run to the AAA titles of the 2010's. It's not a matter of "unless it is able to" but "when it is able to".

bigyabai  5 hours ago

It won't. Laya is a finetuned version of Google's BeRT model, which is almost 10 years old right now.

If BeRT had any potential to disrupt the datacenter buildout, it already would have.