Comment by igorkraw
2 years ago
It should be noted that
1. this is nothing that should surprise anyone who has an intuition on control theory and the evolution of unconstrained markov chains
2. there appear to be relatively easy mitigations https://news.ycombinator.com/item?id=41061085 (made a separate post because it might be of independent interest to discuss)
3. you still won't get beyond the imititation game boundary without exploration & feedback, i.e. the recursive improvement doomers are, as of now, still wrong
> 1. this is nothing that should surprise anyone who has an intuition on control theory and the evolution of unconstrained markov chains
You don't even need to know what a markov chain is. It is intuitively obvious to anyone with two brain cells to rub together that AI can't improve by eating its own vomit.
I've been telling people this for the past few years. They would like to find out the hard way what control theorists already know.