Comment by esafak
1 hour ago
> I think so. This machine might produce answers we value, but it would not, in itself, produce human understanding of those answers.
It's nice that the author is optimistic, but won't the AI be best placed to dumb down its increasingly complex proofs into a language us lowly humans can understand? To keep thinking until it can refactor complex proofs into ones from 'the book'?
As they go on to explain, a human understandable proof is different than a human actually understanding the proof. That actual human understanding (like, in a brain of a human) is one of their stated goals.
Producing human understandable proofs is possibly a job best for humans today, but the author appears to agree with you that this is probably fleeting (and argues that even if you disagree, it should probably be treated as if it is fleeting when planning for the future):
> Right now AI systems arguably underperform us at theory-building, asking questions, exposition, … so we could prioritize and reward those skills. I think this is unwise: compare the speed at which the academy adapts to the speed at which model capabilities improve. We need to consider the endgame. If the models remain incapable in some domain, we can adjust later.