← Back to context

Comment by nine_k

7 days ago

I hope this is the future. Offline, small ML models, running inference on ubiquitous, inexpensive hardware. Models that are easy to integrate into other things, into devices and apps, and even to drive from other models maybe.

15 comments

nine_k

Reply
WhyNotHugo  6 days ago

Dedicated single-purpose hardware with models would be even less energy-intensive. It's theoretically possible to design chips which run neural networks and alike using just resistors (rather than transistors).

Such hardware is not general-purpose, and upgrading the model would not be possible, but there's plenty of use-cases where this is reasonable.

  • amelius  6 days ago

    But resistors are, even in theory, heat dissipating devices. Unlike transistors, which can in theory be perfectly on or off (in both cases not dissipating heat).

  • regularfry  5 days ago

    It's theoretically possible but physical "neurons" is a terrible idea. The number of connections between two layers of an FF net is the product of the number of weights in each, so routing makes every other problem a rounding error.

  • divamgupta  6 days ago

    The thing is that the new models keep coming every day. So it’s economically not feasible to make chips for a single model

theshrike79  6 days ago

This is what Apple is envisioning with their SLMs, like having a model specifically for managing calendar events. It doesn't need to have the full knowledge of all humanity in it - just what it needs to manage the calendar.

depingus  6 days ago

Hmm. A pay once (or not at all) model that can run on anything? Or a subscription model that locks you in, and requires hardware that only the richest megacorps can afford? I wonder which one will win out.