Comment by Fernicia
3 days ago
His point is that humans are prone to the same error. The flooded engine damage doesn't come from humans recognising the danger of a flooded road and choosing not to attempt it.
3 days ago
His point is that humans are prone to the same error. The flooded engine damage doesn't come from humans recognising the danger of a flooded road and choosing not to attempt it.
Im responding to the implication that you "have to be ok with good enough" and that somehow this will be a mostly fine autonomous experience with this
> A car that only fails in a road conditions edge case is good enough for the vast majority of cases. You accept that, and issue a manual override for when that edge case pops up
But its just like LLMs. They will never be perfect, and so if you arent actively paying attention and steering the behavior then there is always a risk of spectacular failure. Because if you arent paying attention to "needing to [apply a] manual override" then all of a sudden the AI has `rm -rf /` and you had it in "bypass permissions" mode.
No one cares about flooded engines that Google has to pay for. They care about a taxi that might kill them.
You have to compare this to the number of taxi and Uber drivers who will drive into moving water with passengers on board while a passenger is telling them to stop.