Comment by moffkalast
4 days ago
Yeah cause every car needs 8xH200 pulling 10kW to run a VLM at realtime speeds. Would be unfortunate if 4G dropped out under some trees while using the API after all.
4 days ago
Yeah cause every car needs 8xH200 pulling 10kW to run a VLM at realtime speeds. Would be unfortunate if 4G dropped out under some trees while using the API after all.
Power usage isn't an issue. 10 kW is 13 HP. The size, price, and fragility of the components is the issue.
Steady 10KW load means 40 less miles after an hour of driving if your EV gets 4mi/kwh. That kind of draw would use up nearly 1/6th of my EV's battery in an hour.
Man, EVs are very efficient. I was thinking from the perspective of my 120 HP Hyundai Venue, where it would be only an extra 10% of peak power.
1 reply →
GPUs/XPUs are small and solid state so it’s only really price that’s a huge liking factor.
And the disinclination of these companies to push the weights of their cutting edge models into people’s cars where they can be dumped.
> Yeah cause every car needs 8xH200 pulling 10kW to run a VLM at realtime speeds.
When the models stop improving, we will get model-specific ASICs that are much more power-efficient.
> When the models stop improving
Soo, never? Granted Cerebras is a thing, if the process can be commoditized.
At the moment the area of edge inference at speed seems pretty bleak though.