Comment by andai

I have an example from 2023, when Auto-GPT (think OpenClaw but with GPT-3.5 and early GPT-4 — yeah it wasn't great!) was blowing up.

Most people were just using it for the same task. "Research this stuff and summarize it for me."

I realized I could get the same result by just writing a script to do a Google search, scrape top 10 results and summarize them.

Except it runs in 10 seconds instead of 10 minutes. And it actually runs deterministically instead of getting side tracked and going in infinite loops and burning 100x as much money.

It was like 30 lines of Python. GPT wrote it for me.

My takeaway here was, LLMs are missing executive function. The ability to consistently execute a plan. But code runs deterministically every time. And - get this - code can call LLMs!

So if your LLM writes a program which does the task (possibly using LLMs), the task will complete the same way every time.

And most of the tasks people use LLMs for are very predictable, and fit in this category.

People are now repeating the exact same thing Auto-GPT thing with OpenClaw. They're using the slow, non-deterministic thing as the driver.

It actually kinda works this time — it usually doesn't get stuck anymore, if you use a good model — but they're still burning a hundred times more money than necessary.