Comment by polyglotfacto2

> Some prior studies that find speedup do so with developers that have similar (or less!) experience with the tools they use. In other words, the "steep learning curve" theory doesn't differentially explain our results vs. other results.

I think one would have to compare the difficulty level of tasks.

I speculate that on easy tasks, LLM's can do a great job based on their training data alone, so you'd experience a speedup regardless of your prompt engineering skill level. But on large codebases and for complex tasks, an LLM cannot stand on it's own legs, and the differentiator becomes the quality of the prompt.

I think you'd need not only expert programmers, but expert programmers who have become expert prompt engineers(you would need some kind of extensive system prompt describing how the large codebase works), and those don't really exist yet, I think.