Comment by kazinator

> How much of this is just "AI is bad at everything"

"AI Language Models" are not touted as some general AI that is smart at everything, like a clever person with multiple intellectual skills integrated into one.

AI language models are for modeling language, not for math problem solving, or anything else. People good at language aren't always good at math.

DeepL produces very good, correct translations for "Alice has five more balls than Bob, who has two balls after he gives four to Charlie. How many balls does Alice have?" into numerous languages, even though it doesn't offer a solution.

I have little doubt that an AI system could be trained to translate word problems like this into systems of equations, which could be dumped into a some decades-old CAS to obtain a solution, which the AI could map back into the verbal domain through the identities between the math variables like x and Alice's apples.

"Hey look, that human who is supposedly good at math can't produce a painting of the Grand Canyon in the style of Monet, even if given eight months to do it, and is easily defeated in chess."