Comment by Vecr
3 years ago
It's the best option, even assuming no one raced and you got full funding and enthusiasm, it would be plausible to put success at 40%. That's after a generation of work and trillions of (2022, probably) dollars, but it's sure better than the ~0% chance I'd give anything else. The coordination problem for getting AI using this method and the coordination problem to prevent anyone getting AI at all is quite similar though, but probably somewhat more preferable.
> It's the best option
That's not an answer.
> it would be plausible to put success at 40%.
No, it wouldn't. You just made up that number based on nothing.
We have a good amount of experience with provability in software, and its limitations are well-known. Tegmark et al. seem to me just to be waving the magic pixie dust of "AI will solve this", and making unlikely claims, with no real substance.
I was asking what about what they're saying makes you think that "it's the best option", but it's clear I'm not going to get a useful answer from you.
If it's not the best option, what is? It's the only method where I think you would be able to detect failures before the AI system is running. RLHF and "Constitutional AI" absolutely can't do that with current neural network analysis methods.