Comment by drakenot
9 hours ago
Are LLMs going to eventually become the architecture that powers self-driving cars?
Seeing them play Portal and other video games, I'm curious if they will eventually help solve that last N% of self-driving.
9 hours ago
Are LLMs going to eventually become the architecture that powers self-driving cars?
Seeing them play Portal and other video games, I'm curious if they will eventually help solve that last N% of self-driving.
A VLM (a vision-language model) is already being used by Waymo[1]. It's useful for scenarios that require reasoning and general knowledge.
[1] https://waymo.com/blog/2025/12/demonstrably-safe-ai-for-auto...
Fast forward to me sitting at a Green Light waiting for my usage to reset for the week so I can get to where I’m going
In a sense, they already are. The giant leap in self driving cars we've seen in the last handful of few comes from using transformer models.
I remain worried about prompt injection style attacks against self-driving cars.
Imagine if someone finds a weird image pattern that gets misinterpreted as instructions and hangs that off a bridge over a freeway.
It'll get rooted over the uplink/WiFi/BT long before that. Probably even more likely for non-SDVs.
It's not a coincidence that Waymo started becoming viable after GPT-3.
News to me. Where did you learn about that?
GPT-6 is multimodal, LLMs alone have no vision capability
"LLM" is now in practice a superset of "LMM"
To my understanding gemma is shipped with waymo, in a highly modified fashion.
I'm skeptical, tho. Cost will push for right sizing, much like we have right sized a lot of things about modern cars.