Comment by jxy

> “When multiplying really large numbers together … they’ll forget to carry somewhere and be off by one,” says Vineet Kosaraju, a machine learning expert at OpenAI. Other mistakes made by language models are less human, such as misinterpreting 10 as 1 and 0, not ten.

So the expert has never seen a seven year old struggling in adding two single digit numbers together? Did the expert learn 1 and 0 being 10 first and learn to speak second?

> The MATH group found just how challenging quantitative reasoning is for top-of-the-line language models, which scored less than 7 percent. (A human grad student scored 40 percent, while a math olympiad champ scored 90 percent.)

Is this that surprising? How would our ieee editor score on the same problem set?