Comment by jvanderbot
4 days ago
You might be interested to learn that the bitter lesson has already been grok'd by generations of autonomous car company engineers, and many or all have incorporated learned components (at minimum) in all their vehicle stacks.
There's also a very tangible limitation of the bitter lesson.
If, over time, compute climbs, and so compute-bound data-driven general architectures beat bespoke architectures (this is the bitter lesson), then it is not necessarily true that the most general architecture now beats all available bespoke architectures now (or even in the near/mid future - the crossover point is "eventually").
Bitter lesson is most tangible for long-running research directions. Sometimes you need something working as best as possible now.
Yeah. Every major self driving model that I’m aware of is fully e2e at this point. Going from fused sensor output to control+debug vectors.
This is more generalised.
But also since there’s a huge volume of data it’s too expensive to just keep scaling compute up (per car overhead) so there are necessary tricks involved.
I do think having a large model that can do this means that a small specialised model could be distilled form it though. Which is probably the most feasible path to production IMO.
This seems like it would be way too dangerous for all sorts of security reasons.
A later entrant can potentially side step those investments if their now is later. Since self driving car ventures aren’t profitable yet and need to make up their investments over time, thats a real risk for them.
But what might happen imo, is that these huge models might be much better at learning from data in an unsupervised manner, so a model based on Astra might become a much better driver in a short period of time (and given a smaller set of training data) - this is due to it understanding much more of the inputs, and being able to draw conclusions from it much more efficiently, thus the information available for training it is greater per sample.
Then the big model can teach a small model to become almost as good a driver. This might be substantially more efficient way to train stuff, and might be fairly quick and straightforward.
In practical terms, I feel this means we can see huge jumps in capability overnight. And this is a general indicator of AI progress, not only in this narrow scope.
You're making a "might" argument for a more general, more CPU/Data intensive process, which I just call "bitter lesson". I wouldn't even say "might" i'd just say "yeah, eventually so!"