Comment by cs702
3 years ago
I've yet to see how Waymo and other self-driving systems perform in open-ended testing, outside tightly restricted, geofenced environments.
Otherwise, I agree that Tesla FSD Beta has been progressing nicely. I don't know if it will take 1, 2, or 5 years to get FSD Beta to an acceptable rate of graceful failures, but I agree it looks likely to get there before the end of the decade!
I've yet to see Tesla FSD Beta perform in any meaningful operational environment, no matter how restricted, within a factor of 100x of Waymo or Cruise, and Waymo and Cruise are still at least a factor of 10x off of "equal to human drivers". Superiority can not be claimed if a system is unacceptably bad in all circumstances (i.e. 1,000x worse than the minimum acceptable standard for real commercial usage) and worse in every case than a comparable alternative. It only makes sense if the system is better at something other than being allowed to create unacceptably bad results in a more diverse range of circumstances.
Tesla has not focused on any individual area to any large extent so why would you measure them on that? That not the problem they are trying to solve and its not the problem that worth investing 30 billion for.
> within a factor of 100x of Waymo
What's your evidence for this? Is there anybody who has done a systematic comparison of Tesla performance in Arizon zone of Waymo?
California has a reporting requirement for all companies testing L4/5 vehicles:
https://www.dmv.ca.gov/portal/vehicle-industry-services/auto...
Here is a more human digestible summary:
https://thelastdriverlicenseholder.com/2022/02/09/2021-disen...
In 2021, Waymo averaged ~7,900 miles per disengagement and Cruise averaged ~41,000. In 2020, Waymo averaged ~30,000 and Cruise around 28,500.
Tesla is absent from those reports as they have deliberately declared that all of their vehicles do not even qualify as L4/5 autonomous vehicles. They have furthermore not released any 3rd party auditable metrics for any of their claims. So, from an official perspective, Tesla is infinitely worse than Waymo.
From an unofficial standpoint, we can use Youtube videos and self-reported tracking by invested fans such as here https://www.teslafsdtracker.com/
Both of those classes of unofficial metrics by positively biased groups consistently demonstrate around 10-20 miles per disengagement at max.
As Waymo averaged 8,700 last year, that makes Tesla around 400-800x worse than Waymo as of last year and around 2,000-4,000x worse than Cruise as of last year.
We can also see from the unofficial Tesla fan metrics that FSD Beta has seen no material improvement from around one year ago.
>I've yet to see how Waymo and other self-driving systems perform in open-ended testing, outside tightly restricted, geofenced environments
But you have seen how Tesla performs in such environments, and you aren't allowed to take your hands off the wheel.
What makes you assume Tesla has the right approach and the other have companies have to be measured against it?
I never said that Tesla has the right approach nor that other companies have to be measured against it. What I did say is that I perceive Tesla FSD Beta as now being significantly ahead, based on the beta-tester videos I've been seen online. BTW, a year ago, I wouldn't have said that about Tesla FSD Beta.