Comment by thymine_dimer
12 hours ago
Doesn't this just suggest that the next frontier for powerful AI models is to ask challenging questions, not simply solve them?
Terry even says this: "In fact, it is now the identification of a promising problem which is the scarce and precious resource."
The creativity and insight needed to ask a question that Terry gets excited about is the next step. Perhaps OpenAI should create a set of challenging questions and offer a prize to solve them.
The incentives are massively skewed towards the AI labs investing their massive amounts of compute into being the first to solve an outstanding problem.
It's a marketing game for them, any societal benefits are secondary. Winning a prize is going to get headlines and feed into the "AGI soon, machine replaces another career" narrative they crave unlike coming up with some (possibly) interesting problems.
I think the problem with AI asking questions is that it will ask questions that are interesting to it but not necessarily us. AI, as a model, will never be a perfect copy of a human. It will always be a simulation, and thus to some extent, will ask questions that humans find irrelevant and solve problems that humans find irrelevant.
For anyone facing an existential crisis on AI, your ace in the hole is your humanity. Only you have it, and only you will be the best judge of what is good and interesting (to a human at least).
>your ace in the hole is your humanity
Average HN Poster: [nervous sweating]
Exactly.
My humanity is not paying my bills.
If AI can generate questions and then answer them, what are the people for?
If humans can shovel dirt, then what are the ants for?
https://en.wikipedia.org/wiki/Decline_in_insect_populations
> ask challenging questions
As far as I can tell, it's still not possible for an agent to reliably determine if a question is a good question. That means the test part of the loop cant be fulfilled.