Comment by monitron

Precisely! I don't think this is the whole problem but it's a part of it I keep coming back to. We took the one thing that computers were good at (getting the same answer every time, quickly and efficiently) and tossed it out the window.

Good AI system design can help somewhat, but even if you give the LLM a calculator tool, it's not guaranteed to use it every time, or to write the tool use correctly, or to copy out the answer correctly.