Comment by geraneum
18 hours ago
What a dystopian future where your computer use will be basically “metered” by the token. At least right now we don’t have to pay per mouse clicks.
18 hours ago
What a dystopian future where your computer use will be basically “metered” by the token. At least right now we don’t have to pay per mouse clicks.
I can't imagine it worse that the current hell hole of SaaS, which is basically a way to withhold ownership while forcing you to pay for unwanted feature bloat, unnecessary redesigns, and strict dictation of how you can use the software.
Excel '98 probably covers 90% of excel users needs, but here we are sending Microsoft billions a month to lease excel.
Local inference solves that.
We used to pay for phone calls not long ago. Now a video call is free.
This doesn’t make sense. You mentioned OS and drivers, why send the commands to your hardware through the black box and extra hoops instead of directly doing so?
Do you believe the average person can afford the 1TB+ VRAM required to run a competent model?
The expectation is of course that that will not be required. Competent models will become smaller and the compute to run them will become cheaper, once the market stabilizes. That will take some time of course.
Eventually yes. You can buy greeting cards today for $5 that have more compute power than the Eniac.
Not today. But just like the average person can now afford a smartphone which is more powerful than a workstation not long ago, we will have enough computing power for good enough local inference, at an accessible price.
3 replies →
Yes because the models will be etched straight into silicon.