Comment by gloosx
1 year ago
It's crazy that it just tries to bruteforce it by picking numbers, and in your case it took more steps before concluding a success/failure, which seems quite to be random to me, or at least dependent on something.
What's clear is that it doesn't have any idea about mathematical deduction and induction – a real chain-of-thought which kids learn in 5th grade.
Lots of people don’t either. I think it probably just needs more 5th grade math problems in the rlhf corpus :)
It certainly needs them, but nothing will stop openai from making marketing claims like this today:
"places among the top 500 students in the US in a qualifier for the USA Math Olympiad (AIME)"
Like the top 500 students in the US are just popping random numbers into the problems, lol