Comment by sashank_1509

(Disclosure: I work for Waymo)

While there is plenty of classical robotics code in our planner, I wouldn't want people to assume that we don't use neural networks for planning.

Just because we don't deploy end-to-end models (e.g., sensors to controls), but have separate perception and planning components doesn't mean there isn't ML in each part. Having the components separate means we can train and update each individually, test them individually, inject overrides as needed, and so on. On the flip side, it's true that because it's not learned end-to-end today that there might exist a vastly simpler or higher quality system.

So we do a lot of research in this area, like EMMA (https://waymo.com/research/emma/) but don't assume that our planning isn't heavily ML based. A lot of our progress in the last couple of years has been driven by increasing the amount of ML used for planning, especially for behavior prediction (e.g., https://waymo.com/research/wayformer/)

> basically has painstakingly listed every possible road obstacle, has coded every possible driving logic to it, and ... addressed every edge case

Removed that "manually" world so now it describes exactly what you would have to do to train an end to end neural network.

NNs don't get information from nothing, you would have to subject them to the exact same obstacles, geometries and behaviors you coded on the manual version.

Not every edge case, but enough that the vehicle can correctly determine it doesn't know how to proceed and must ask a human to choose from among a menu of choices. This is how Waymo described how supervision works. Nobody actually drives the vehicle remotely. They just make a decision the on-board intelligence has decided it can't make.

One good bet based on Waymo's decision to expand is that the amount of supervision each robotaxi needs keeps going down, so supervision is not tightly coupled to fleet size.

I think it would be the other way around, academic roboticists are very well aware of how damn hard the physical world is.

IMO the most relevant point is that, even with all that data, Waymos are backed up by a large team of humans that can help guide the cars through "difficult" common-sense situations that AI is simply not capable of handling, both because artificial neural networks are very primitive and stupid compared to vertebrate brains, and because it is practically impossible to collect enough data for the dumb ANNs to learn from. Self-driving companies are very cagey about this.

I suppose the (crummy) analog is that a human's "models" are equally not entirely general; we have evolved a particular architecture that is baked into our hardware and perpetuated via our DNA.

It's fuzzy and plastic and complex, but the brain has functional areas, there is intelligence more local to specific sensors, pipelines where fusion happens, governors and supervisors, specific numeric limits to certain tasks, etc.

This is a bit akin to your "listing every possible item", in a way, in the sense that there are definitely finite structures tuned toward the application of being human.

This interplay via our supposed "AGI" and what is "cached" in our also not static but evolving hardware is really one of the most fascinating aspects of biology.

https://en.wikipedia.org/wiki/Siphonaptera_(poem)

Big edge cases have little edge cases that require their own code / and those edge cases have smaller edge cases with yet more code.

My shorthand is "the real world is a fractal of edge cases".

Link to talks?