Comment by pbohun

As an example, is it faster to multiply 2 numbers using a single MUL instruction or multiply matrices with millions (or billions) of elements multiple times? Multiply that overhead by every single thing you do and that's the overhead of using an LLM to emulate a program.