Comment by seanmcdirmid
3 days ago
As long as your program is large and multi-threaded (most programs that matter commercially), it is not very analyzable or repeatable. You replace those qualities with QA and tests, the same is true with prompting.
3 days ago
As long as your program is large and multi-threaded (most programs that matter commercially), it is not very analyzable or repeatable. You replace those qualities with QA and tests, the same is true with prompting.
Sounds like it's time for your LLM daddy to have the Coq talk with you..
Eve if "write code -> run QA -> analyze failures -> rewrite code" is cheaper for most commercial software than thorough upfront formal verification, it works precisely because the programs are analyzable.
When the code spit out by an LLM does not pass QA one can merely add "pls fix teh program, bro, pls no mistakes this time, bro, kthxbye", cross their fingers and hope for the best, because in the end it is impossible -- fundamentally -- to determine which part of the prompt produced offending code.
While it is indeed an interesting observation that the latter approaches commercial viability in certain areas there is still somewhere between zero and infinitesimal overlap between prompting and engineering.
Think of it this way, some engineers go into people management, they aren’t coding directly anymore…they are managing people that code. Prompting is a similar lateral promotion, just the people you are managing are dumber AIs, you get a lot of them, and instead of meetings you communicate with them via prompts. The fact that they can also do QA is critical because they make a lot of mistakes, but can actually fix those mistakes, so you just devote more AI time to that.
> they are managing people that code. Prompting is a similar lateral promotion
So prompting is a lateral move away from engineering to management? Are we arguing semantics here, because that's quite what I was saying, just in the other direction.
1 reply →