Comment by bluefirebrand

So here's my problem, ultimately

If the task horizon for the LLM is shorter than writing it yourself, this likely means that the task is well defined and has an easy to access answer

For this type of common, well defined task we shouldn't be comparing "how long it takes for the LLM" against "how long it takes to write"

We should be comparing against "how long it takes to find the right answer on SO"

If you use this metric, I bet you the best SO answer, which is also likely the first google result, is just as fast as the LLM. Maybe faster