Comment by zelphirkalt
4 hours ago
Since lidar has distance information and cameras do not, it was always a ridiculous idea by a certain company to use cameras only. Lidar using cars are going to replace at least the ones that don't make use of this obvious answer to obstacle detection challenges.
The reasoning is cynical but sound. If the system uses only the sensing modes people have, it will make the mistakes people do. If a jury thinks "well I could have done that either!" You win. It doesn't matter if your system has fewer accidents if some of the failure modes are different than human ones, because the jury will think "how could it not figure that out?"
Until a lawyer points out other cars see that. My car already has various sensors and in manual driving sounds alarms if there is a danger I seem not to have noticed. (There are false alarms - but most of the type I did notice and probably should have left more safety margin even though I wouldn't hit it)
also regulators gather srastics and if cars with something do better they will mandate it.
I don't think that's the reasoning.
The reasoning was simply that LIDAR was (and incorrectly predicted to always be) significantly more expensive than cameras, and hypothetically that should be fine because, well, humans drive with only two eyes.
Musk miscalculated on 1) cost reduction in LIDAR and 2) how incredible the human brain is compared to computers.
Having similar sensors certainly doesn't guarantee your accidents look the same, so I don't think your logic is even internally sound.
Sensor fusion is also hard to get right, since you still need cameras you have to fuse the two information streams. Thats mainly a software problem and companies like Waymo have done it, but Tesla was having trouble with it earlier, and if you don’t do it right, your self driving system can be less reliable.
> Musk miscalculated on 1) cost reduction in LIDAR and 2) how incredible the human brain is compared to computers.
And, less excusable, ignorant of how incredible human eyes are compared to small sensor cameras. In particular high DR in low light, with fast motion. Every photographer knows this.
3 replies →
IMHO not using lidars sounds like a premature optimisation and a complication, with a level of hubris.
This is a difficult problem to solve and perhaps a pragmatic approach was/is to make your life as simple as possible to help get to a fully working solution, even if more expensive, then you can improve cost and optimise.
Eh, I think ‘miscalculation’ might be giving too much credit about good intentions.
He wanted (needed?) to get on the hype train for self driving to pump up the stock price, knew that at the time there was zero chance they could sell it at the price point lidar required at the time - or even effective other sensors (like radar) - and sold it anyway at the price point that people would buy it at, even though it was not plausibly going to ever work at the level that was being promised.
There is a word for that. But I’m sure there are many lawyers that will say it was ‘mere fluffery’ or the like. And I’m sure he’ll get away with it, because more than enough people are complicit in the mess.
Miscalculation assumes there was a mistake somewhere, but near as I can tell, it is playing out as any reasonable person expected it too, given what was known at the time.
1 reply →
There certainly is a pretty on going miscalculation regarding human intelligence, and consrquentially, empathy.
Seeing the SOTA in FSD techs it is not obvious that Musk made a miscalc so far.
Very recent issue with Waymo https://dmnews.co.uk/waymo-robotaxi-spotted-unable-to-cross-.... This is 17 years after they bet the farm on LIDAR, with no signs its ever going to be cost effective or that it's better than multiple cameras, with millisecond reaction 360 degrees, that never gets tired, drunk, distracted, and also has other cheaper sensors and NN trained on Billions or real world data.
Tesla does not handle rain well either. This is not a LIDAR problem, it is a problem with self driving cars in general.
1 reply →
That's an example of it failing safe. I'd rather it did that than drive me into a sinkhole because it thought it was a puddle.
3 replies →
There is also a report from the same flooding in LA of a Waymo driving into a flooded road and getting stuck.
They might have flipped a switch after that, causing this.
Dude that's not a 'puddle' as the article claims, that's a body of water that it's not even visually obvious whether it's safe to drive through. Maybe I'm a bad driver but I'd hesitate to drive through that in a small car either.
3 replies →
>A vehicle got stuck trying to figure out an obstacle so sensors with less information are better than sensors with more information.
Most accidents happen because people are human, aren't paying attention, are inebriated, not experienced enough drivers, or reckless.
It's not fair to say that vision based models will "make the same mistakes people do" as >99% of the mistakes people make are avoidable if these issues were addressed. And a computer can easily address all those issues
Which means the mistakes vision-based models for today are unique to them.
This is a new and flawed rationale that I haven't heard before. Tesla cameras are worse (lower resolution, sensitivity, and dynamic range) than human eyes and don't have "ears" (microphones).
The cars do have at least one microphone.
Pretty hard to do if your whole selling point is ‘better and safer than human’ however?
As I understand, lidars don't work well in rain/snow/fog. So in the real world, where you have limited resources (research and production investment, people talent, AI training time and dataset breadth, power consumption) that you could redistribute between two systems (vision and lidar), but one of the systems would contradict the other in dangerous driving conditions — it's smarter to just max out vision and ignore lidar altogether.
No, it isn't "smarter." Camera-only driving is the product of a stubborn dogmatic boss who can't admit a fundamental error. "Just make it work"is a terrible approached engineering.
Can hatred of Musk not derail this entire thread please? I have a camera-only ADAS that I think works quite well, but having both would be better.
Why does this matter? You have to slow down in rain/snow/fog anyway, so only having cameras available doesn't hurt you all that much. But then in clear weather lidar can only help.
If your vision is good enough to drive in rain/snow/fog, you don't need lidar in clear conditions. If you planned to spend $10B on vision and $10B on lidar — you would be better off spending $20B on better vision.
2 replies →
Nothing works perfectly in all conditions and scenarios. Sensor fusion has been the most logical approach now, and into the foreseeable future.
Computer vision does not work exactly like human vision, closely equating the two has tended to work out poorly in extreme circumstances.
High performance fully automated driving that relies solely on vision is a losing bet.
> lidars don't work well in rain/snow/fog.
Neither do cameras, or eyeballs.
When it's not safe to drive, it's not safe to drive.
I've been in zero-road-speed whiteout conditions several times. The only move to make is to the side of the road without getting stuck, and turning on your flashers.
Low-light cameras would not have worked. Sonar would not have worked. Infrared would not have worked.
3 replies →
If you have multi-return lidar, you can see through certain occlusions. If the fog/rain isn't that bad, you can filter for the last return and get the hard surface behind the occlusion. The bigger problem with rain is that you get specular reflection and your laser light just flies off into space instead of coming back to you. Lidar not work good on shiney.
Do cameras work well in those conditions? Nope. Also cameras don't work well with certain answer of glare, so as a consumer I'd rather have something over-engineered for my safety to cover all edge cases...
Limited resources? Billions per year are being thrown at the base technology. We have the capital deployed to exhaust every path ten times over.
Even if so, it doesn't mean that capital deployment efficiency and expected payoff make equal sense in all directions.
Then again, it's good that we have self-driving companies with lidar and without — we will find out which approach wins.
The Swiss cheese model would like to disagree.
When you have sensor ambiguity sounds like the perfect time to fail safely and slow to a halt unless the human takes over.
Evidence clearly shows otherwise.
Also, military sensor use shows the best answer is to have as many different types of sensors as possible and then do sensor fusion. So machine vision, lidar, radar, etc.
That way you pick up things that are missed by one or more sensor types, catches problems and errors from any of them, and end up with the most accurate ‘view’ of the world - even better than a normal human would.
It’s what Waymo is doing, and they also unsurprisingly, have the best self driving right now.
This is silly. Cameras are cheap. Have both. Sensors that sense differently in different conditions is not an exotic new problem. The kalman filter has existed for about a billion years and machine learning filters do an even better job.
Cameras are cheap, but, as I understand:
1) it's not cheap to produce lidars at a stable predictable quality in millions;
2) car driving training data sets for lidars are much scarcer (and will always be much scarcer due to cameras' higher prevalence) and at a much lower quality;
3) combined camera+lidar data sets are even scarcer.
2 replies →
I wouldn’t take too much issue with the “cameras are enough” claim if cameras actually performed like eyes. Human eyes have high dynamic range and continuous autofocus performance that no camera can match. They also have lids with eyelashes that can dynamically block light and assist with aperture adjustment.
The appeal to human biology and argument against fusion between disparate sensors kinda falls flat when you’re building a world model by fusing feeds from cameras all around the car. Humans don’t have 8 eyes in a 360 array around their head. What they do have is two eyes (super cameras) on ~180 degree swiveling and ~180 degree tilting gimbal. With mics attached that help sense other vehicles via road noise. And equilibrioception, vibration detection, and more all in the same system, all fused. If someone were actually building this system to drive the car, the argument based on “how did you drive here today?” gets a lot stronger. One time I had some water blocking my ear and I drove myself to the hospital to get it fixed. That was a shockingly scary drive — your hearing is doing a lot of sensing while driving that you don’t value until it’s gone.
Yea, even in the case they could match human level stereo depth perception with AI, why would they say "no" to superhuman lidar capabilities. Cost could be a somewhat acceptable answer if there wouldn't be problems with the camera only approach but there are still examples of silly failures of it. And if I remember correctly they also removed their other superhuman radar in their newer models, the one which in certain conditions was capable of sensing multiple cars ahead by bouncing the signal below other cars.
Because they don't have superhuman LIDAR. They never did. Nobody ever did. LIDAR input is not completely reliable so what do you do then?
Why make things more complicated than they need to be? Humans don't have lidar and we are the only intelligence that can reliably drive. Lidar just seems like feature engineering, which has proven to be a dead end in most other AI applications (bitter lesson).
https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson...
The bitter lesson I think is a great way of explaining the logic behind Tesla's strategy. People aren't getting it.
Whether or not it'll actually work remains to be seen, but it's a perfectly reasonable strategy. One counterargument would be that the bitter lesson can be applied to LIDAR too; it's not exclusive to visual data.
> Why make things more complicated than they need to be? Humans don't have lidar and we are the only intelligence that can reliably drive.
Because we want self driving cars to be safer than human driven cars.
If humans had built in lidar we would use it when driving.
Self driving cars are not equipped with human brains so this doesn’t really make sense.
“We should achieve self driving cars via replicating the human brain” strikes me as an incredibly inefficient and difficult way to solve the problem.
> we are the only intelligence that can reliably drive.
Science would like to point out that rats also can learn to drive
https://theconversation.com/im-a-neuroscientist-who-taught-r...
Humans can drive with eyes only, but we are better drivers when we can also use other senses like hearing. If humans has lidar we would use it when driving.
Don't cars already use a ton of sensors that don't reproduce human senses and ways of doing things?
This knee-jerk reply is old and tired, and the counterarguments are well-trod at this point. Even if cameras-only can build a car that’s as good as humans, why should we settle for “as good as“ humans, who cause 40,000 fatalities a year in the US? If we can do better than humans with more advanced sensors, we are practically morally obligated to do that.
I'm not an expert on ML vision, but I do have a Tesla and it seems to be able to tell how far away things are just fine. I'm not sure what would be wrong with the vision system that lidar needs to fix.
The phantom braking issue with auto pilot tells me it can’t. A shadow from a tree doesn’t trigger your brakes locking up at 70+ mph when there’s a lidar sensor to tell you it’s not a physical object.
“Just buy FSD” isn’t a reasonable answer to a problem literally no other automaker suffers from.
Stopped using autopilot because of the phantom braking.
It's also recently gotten much worse at lane departure sensing, often confused by snow or slightly faded road markers. Not pleasant to have the alarms go off while calmly and safely driving.
Yeah it's BS. Tesla uses lidar where it makes sense: They have a small lidar fleet to collect ground truth depth data for better vision estimation. This part is long solved.
considering cameras can create reliable enough distance measurements AND also handle all the color reception needed for legally driving roads it was always a ridiculous idea by a certain set of people that lidar is necessary.
No, cameras cannot create reliable distance measurements in real-world conditions. Parallax is not a great way to measure distance for fast, unpredictably moving objects (such as cars on the road). And dirt or misalignment can significantly reduce accuracy compared to lab conditions.
Note that humans do not rely strictly on our eyes as cameras to measure distances. There is a huge amount of inference about the world based on our internal world models that goes into vision. For example, if you put is in a false-perspective or otherwise highly artifical environment, our visual acuity goes down significantly; conversely, people with a single eye (so no parallax-based measurement ability) still have quite decent depth perception compared to what you'd naively expect. Not to mention, our eyes are kept very clean, and maintain their alignment to a very high degree of precision.
Stereo cameras are useless against repeating patterns. They easily match neighboring copies. And there are lots of repeating or repeating-like patterns that computers aren't smart enough to handle.
You can solve this by adding an emitter next to the camera that does something useful, be it just beaconing lights or noise patterns or phase synced laser pulses. And those "active cameras" are what everyone call LIDARs.
'cameras can see in color, therefore lidar is unnecessary for self driving' is unconvincing
There are tons of evidence showing that cameras are alone are not safe enough and even Tesla has realized that removing lidar to save cost was a mistake.
Just say Tesla, why censor yourself.
I have a suspicion here on HN. When criticizing big tech, especially Google and FB, at a certain time of the day a specific cohort comes online and downvotes. Suspiciously, that is a time when one could conclude, that now people in the US start working or come online. Either fanboys, employees or an organized group of users trying to silence big tech criticism.
I have no proof of course and it might be coincidence, or just difference of mindset between US citizens and Europe citizens. It happened a few times already and to me looks sus.
But if they actually read and not just ctrl+f <company name>, then of course not writing the company name, but hinting at it in an obvious way is no more helpful either.
I have seen this happening multiple times, some to fairly reasonable comments with a just tiny negative tone.
There is also flagging abuse which effectively kills the comment /post.
I know for a fact at least 1 bigger US company has a bot in slack that brings up any mentions of $companyname on hackernews...
It's been my experience that hn and reddit have a very high overlap in audience these days. The jerrybreakseverything crowd. Anything anti-tesla, anti-grok, is applauded.
1 reply →
> Since lidar has distance information and cameras do not, it was always a ridiculous idea by a certain company to use cameras only
Human eyes do not have distance information, either, but derive it well enough from spatial (by ‘comparing’ inputs from 2 eyes) or temporal parallax (by ‘comparing’ inputs from one eye at different points in time) to drive cars.
One can also argue that detecting absolute distance isn’t necessary to drive a car. Time to-contact may be more useful. Even only detecting “change in bearing” can be sufficient to avoid collision (https://eoceanic.com/sailing/tips/27/179/how_to_tell_if_you_...)
Having said that, LiDAR works better than vision in mild fog, and if it’s possible to add a decent absolute distance sensor for little extra cost, why wouldn’t you?
Human/animal vision uses way more than parallax to judge distances and bearings - it uses a world model that evolved over millions of years to model the environment. That's why we can get excellent 3D images from a 2D screen, and also why our depth perception can be easily tricked with objects of unexpected size. Put a human or animal in an abstract environment with no shadows and no familiar objects, and you'll see that depth perception based solely on parallax is actually very bad.
Human eyes are much better than cameras at dealing with dynamic range. They’re also attached to a super-computer which has been continuously trained for many years to determine distances and classify objects.
I don’t like the comparison between humans and humans. Humans don’t travel around at 100mph in packs of other humans. Why not use every sensor type at our disposal if it gives us more info to make decisions? Yes I understand it’s more complicated, but we figure stuff out.
Let me know when you have a camera package with human eye equivalency.
I'll preface by saying lidar should be used with autonomous vehicles.
Individual cameras don't have distance information, but you can easily calibrate a system of cameras to give you distance information. Your eyes do this already, albeit not quantitatively. The quantitative part comes from math our brains aren't setup to do in real time.
It was cost wasn't it?
If this lowers Lidar costs, and Tesla has spent all this time refining the camara technology. Now have both.
Use both.