Comment by tjoff
3 hours ago
the whole concept of having cars with that are almost capable of self-driving is utterly insane, the very least you'd need special training. We are not equipped to deal with something that works brilliantly most of the times but might kill you for no foreseeable reason.
Yeah, that's definitely the frustrating zeitgeist for me these days, which extends to LLMs.
I'd also emphasize that it's not just error rate, but the shape/distribution of errors, and our (in)ability to build control systems around them.
To illustrate, imagine if someone unveiled a car which was unambiguously safer in every statistical measure... buuuut some of its unsafety came from jumping the curb to chase and kill pedestrians, under circumstances we can't predict for reasons we can't diagnose.
Even pilots are getting this automation-complacency now. A plane crashed because the military jammed GPS and the pilot wasn't used to any backup procedures even though 30 years ago they were the only procedures.