Comment by voidUpdate
5 hours ago
People keep talking about how LLMs are like a compiler from human language to code. We commit source code instead of just compiled machine code, so why should this be any different? The "source code" is the prompts
5 hours ago
People keep talking about how LLMs are like a compiler from human language to code. We commit source code instead of just compiled machine code, so why should this be any different? The "source code" is the prompts
The prompt isn't very useful. You'd see the exact same prompt on every ticket for me.
Prompt 1: "Research <X> domain, think deeply, and record a full analysis in /docs/TICKET-123-NOTES.md"
Prompt 2: Based on our research, read TICKET-123 and began formulating solutions. Let's think this problem through and come up with multiple potential solutions. Document our solutions in TICKET-123-SOLUTIONS.md
Prompt 3: Based on Solution X, let's formulate a complete plan to implement. Break the work into medium sized tasks that a human could complete in 5-10 hours. Write our plan in TICKET-123-PLAN.md
I've often thought that some of this metadata, such as the research, solutioning and plan could be shared. I think they're valuable for code review. I've also translated these artifacts into other developer documentation paradigms.
But the prompts? You're not getting a lot of value there.
> Prompt 1: "Research <X> domain, think deeply, and record a full analysis in /docs/TICKET-123-NOTES.md"
> Prompt 2: Based on our research, read TICKET-123 and began formulating solutions. Let's think this problem through and come up with multiple potential solutions. Document our solutions in TICKET-123-SOLUTIONS.md
> Prompt 3: Based on Solution X, let's formulate a complete plan to implement. Break the work into medium sized tasks that a human could complete in 5-10 hours. Write our plan in TICKET-123-PLAN.md
Sounds to me that all these 10x - 100x "engineers" can be removed from the loop.
Almost! We are certainly on the precipice of the vast majority of white collar work being removed from the loop.
However, what each domain will tell you (engineering included) is that AI doesn't understand the full context of what you're doing and the point of the business and where to spend effort and where to cut corners. There is definitely still room for competent engineers to iterate here on the solutioning and plans to refine the AI work into something more sturdy.
Although this is only in domains where code quality truly matters. A lot of consumer software without SLA's are just vibe coding full speed now. No code review, AI writing 100% of the code.
5 replies →