Comment by HarHarVeryFunny
13 hours ago
FWIW while Karpathy was at Tesla he was basically working on the vision component. The actual driving component (using vision as an input) was originally all C++. They may have started migrating parts of the driving component from C++ to neural networks while Karpathy was there, but most of it happened after he left in 2022, with the big switch being FSD 12 in 2024. User reports from before/after FSD 12 are like night and day.
I was never convinced by the "vision only" approach - I don't see the point of throwing out or refusing to have additional data from other sensors.
I suppose that with modern ML they can just toss it in the blender and reap the benefits ...
I'm not talking about vision (cameras) vs lidar etc, just the Tesla FSD architecture that separates the "vision" component (turning camera/sensor inputs into symbolic road/sign/vehicle/pedestrian/etc data), and the driving component which takes the vision data, plus current location and destination, and uses that to actually drive the car - switch lanes, make turns, avoid obstacles etc etc.