Comment by nichochar
3 years ago
[Disclaimer] I worked for Cruise for 4 years.
I agree with everything you said, but chuckled at this particular part (which is very wrong): > Tesla is significantly ahead of everyone else.
Everyone in the industry knows that Tesla is nowhere near the tip of the technology. What Tesla does is _fantastic marketing_. Their whole self-driving division is just a mechanism to sell more cars.
At a high level, this is why:
- The hard thing about self driving isn't the first 95%, it's the impossibly long tail of the last 5% with unique, chaotic and rare scenarios (think, a reflective citern tank with a reflection of the back of a truck transporting stop signs, or terrible weather illusions with fog).
- Doing well on the last 5% is where most of the energy from Waymo/Cruise goes (the two leaders by quite a margin).
- Tesla is camera only. Weather alone means you can't reach critical safety because of this. Cameras don't do fog well, precipitation well, or sunsets/bad lighting well (see many Tesla crashes on freeways bc of this)
- Tesla does well on the 95% and Elon is a marketing genius, with those 2 things it's easy to convince outsiders that "Tesla is significantly ahead of everyone else".
My prediction: before the end of the decade, cruise and waymo have commoditized fleets doing things that most people today would find unbelievable. Tesla is still talking a big game but ultimately won't have permits for you to be in a Tesla with your hands off of the wheel.
edit: formatting and typo
My favorite case so far of the '5%' that you mention happened on my Tesla irt:(object recognition) and I still laugh about it to this day.
I was driving down the road as normal, 4 lane divided highway that's a bit hilly. Suddenly my car starts having what I can only describe as a panic attack saying I'm running a stop sign and blaring alarms.
It was detecting a giant 40ft tall red circle sign a bit away as a stop sign...
That's interesting, because my car never has blared an alarm for that. I live in a part of the country where, in certain parts of semi-rural and hilly areas, they've decided to introduce 4-way stops instead of red lights, or to slow down traffic on a straightaway. As a result, every so often I would not realize there's a 4-way stop (luckily google maps now shows them) and stop a little late. This happened 2-3 times in the last year or two and never did the Tesla make a peep.
Replying to my own comment because I just remembered this as well...
I definitely saw a case of them overfitting their neural networks lately.
Going over a single lane bridge that's an exit ramp, the car started decoding the "other side" of the concrete barrier as oncoming traffic lanes...when there was nothing there.
Excellent example: "reflective cistern tank with a reflection of the back of a truck transporting stop signs"
People tend to forget just how hard the edge cases in vision are!
It’s weird to me people forget how bad vision can be for driving and Exocet Tesla to somehow be better then our own eyes.
How often a do you encounter situations like bad fog/sun set/rain at night where it’s a total struggle to drive and you slow right down to a crawl and even then only do alright because of a ton of inference?
i think Tesla deciding to go vision only will be regarded of one of the greatest blunders is self driving history.
The counterargument to this is that since humans reach acceptable safety levels with vision only, it must be possible to do self driving with vision only. That said, augmenting vision with other methods does seem like a no brainer for better performance.
19 replies →
Or perhaps it will be their advantage in the short term.
Imagine a foggy condition that causes a 50 car pile up on the highway. Which is more likely to avoid the collision, a Tesla that slowed down because it couldn't see or a Waymo/Cruise blasting down the highway at 65 mph because it's Lidar can see through the fog?
5 replies →
As someone who also follows the progress online and watch the presentations by Cruise, Tesla, etc, I agree that Cruise is well ahead of Tesla.
It feels like Tesla’s main strategy is to add more data, more compute power, more simulation, and hope for “convergence”. Maybe that will work, but right now it feels like Cruise’s technology feels more mature and thought through.
Is it? Remarkably Tesla’s compute bill does not seems to be growing exponentially with the amount of data it is supposedly collecting.
Point is I would not take anything Elon says at face value. He’s a marketer who is constantly bending the truth.
Have you actually watched the presentations? They are not the NSA storing all data from all cars on a giant hard-drive.
They are deploy code to cars to search for potential interesting, take lots of sample, curate and create a test and training set from that data.
Also how do you know what the Tesla compute bill is? Given their investment in GPU clusters and their own development of Dojo their cost may well have grown exponentially.
> He’s a marketer who is constantly bending the truth.
He is also a CEO of two major companies that have a good track record of achieving interesting technology and growth. So just denigrating him to a 'marketer' says more about yourself.
They’re not though.
As of today I can’t buy a production car with Cruise. You don’t get points for building something that’s theoretically superior but not an actual product. It’s the same story with companies like Apple that wipe the floor of wannabe hardware companies with theoretically better specs. Like Apple, Tesla actually ships.
Cruise is going the uber route for their service. The robo-cars are already available in some areas
3 replies →
What happens if you drop a Cruise car at a random place in North America it has never seen before?
I believe that Waymo and Cruise have more capable platforms that can more accurately measure the environment at those boundary conditions, where as Tesla's camera only approach more closely mimics human perception. But where do they stand in terms of datasets used for training their models?
It seems like Tesla has a huge advantage in terms of training data by leveraging a fleet of millions of vehicles.
Which is more valuable, experience or technology?
Having ridden the self driving vans from Waymo in Arizona several times, it really does feel like stepping into the future. Although they only cover a specific geographical area, they have really refined the riding experience within it.
George Hotz is 'in the industry' and seems to disagree with you. I have heard others say the same. It seems people who work at Waymo/Cruise are totally convinced by those things.
Different approaches lead to different paths to solutions, I am not convinced that either will be successful and not convinces Waymo/Cruise is ahead.
Unlike those, Tesla actually makes money and uses the technology stack in more limited forms.
If Tesla wants to be serious about self driving cars they should be seeking regulatory approval and running pilot programs with test drivers. Without regulatory approval it’s just a toy, albeit an incredibly dangerous one.
95% is a ridiculous exaggeration. How well can these cars do in the winter? You know, that season we have that can easily keep the ground covered with snow and/or ice for 30% of the year in many cities. I lived in Chicago and it was genuinely difficult to drive for 3-4 months of the year. How well can these cars do during the very rainy hurricane season in Florida?
Do people who work in this industry actually think they've solved 95% of driving scenarios because their software can manage driving in sunny California, Nevada, and Arizona?
> Everyone in the industry knows that Tesla is nowhere near the tip of the technology. What Tesla does is _fantastic marketing_.
Between one and two years ago, that was my perception too. But the rapid progress I've seen with Tesla FSD Beta over the past couple of years, and over the last year in particular, has forced me to change my mind. (Note: I'm talking only about Tesla's beta software. Tesla's production software is behind by dozens of versions and is without a doubt technologically inferior to Waymo and Cruise.)
Less than two years ago, I would have said FSD Beta could only deal with the "first 95%" too. Now, my perception is that FSD Beta routinely handles the first > 99% and fails only on < 1% of situations. Moreover, the failures have become more graceful -- e.g., the car will stop at intersections perceived as risky and ask the driver to confirm go-ahead by pressing the accelerator. If FSD Beta continues to improve, sooner or later it will cross the threshold at which it becomes safer than most human drivers.
Of course, IF I see new evidence that contradicts my perceptions, I'll change my mind again. There's no shame in changing our minds when the facts disagree with our views. FWIW, I'd love to see videos of Cruise and Waymo vehicles, filmed by tens of thousands of regular consumers driving autonomously on fully unrestricted roads, with zero editorial input from Cruise or Waymo.
With cost of lidar plummeting, I wonder how long it will be before Musk changes his mind.
Musk over and over again has shown that if something that he wants is expensive he will invest as much money as required in it to develop and to build it in large quantities. If Musk had thought lidar cost was the big problem, they would be building a lidar giga-factory right now. But they don't because not using lidar wasn't about cost.
He might be right or wrong about that, but he wouldn't just ignore lidar because its currently on on the market cheaply.
Tesla is far ahead on their ability to train a myriad number of heavy AI models (computer vision) at the same time.
That asset will probably be worth far more than the self driving system.