Comment by stevenhuang

That even arithmetic is hard for language models may suggest language models (at least in their current form) are not the most optimum for the task.