I’m critiquing the term “vibe engineering” by highlighting that because of the random, chaotic, black box nature of LLMs it is very difficult to distinguish valid techniques from superstitions.
Thus what you are doing is closer to pigeons bobbing their heads to attempt to influence the random reward machine than it is to engineering.
For example saying things like “It is critical that you don’t delete working code” might actually be a valid general technique, or it might have just been something that appeared to work because of randomness, or it might be something that is needed for current models but won’t be necessary in a few months.
The nature of LLMs makes correctly identifying superstition nearly impossible. And the speed with which new models are released makes trying to do so akin to doing physics in a universe where the laws of nature are constantly changing.
You’re an alchemist mixing gunpowder and sacrificing chickens to fire spirits, not an engineer, and for the foreseeable future you have no hope of becoming an engineer.
I’m also highlighting the insanely addictive nature of random rewards.
See this is where I think I fundamentally disagree. It’s more like being a scientist who takes 10 rats and tries to train them to sniff out TB, and once they find one that can do it reliably, they go and do real work with it. Just because the rat is the one doing the sniffing doesn’t mean the scientists aren’t real scientists. And just because we work with a tool that isn’t perfectly reliable doesn’t make us any less engineers as long as we manage and take ownership of the output as we alter and massage it from the machine.
Imagine back in the day calling devs that didn’t write in assembly “alchemists” because they didn’t fully understand how a compiler works and just trusted it instead. As long as they’re reading the code, testing the logic, and working towards an outcome they take ownership of, they’re engineers, and so are people who use LLM’s. It’s dangerously close to elitism gatekeeping to say otherwise.
Also I’ve been a software engineer for 15 years so I think I don’t “have no hope of becoming a software engineer”, no personal attacks please.
I’m critiquing the term “vibe engineering” by highlighting that because of the random, chaotic, black box nature of LLMs it is very difficult to distinguish valid techniques from superstitions.
Thus what you are doing is closer to pigeons bobbing their heads to attempt to influence the random reward machine than it is to engineering.
For example saying things like “It is critical that you don’t delete working code” might actually be a valid general technique, or it might have just been something that appeared to work because of randomness, or it might be something that is needed for current models but won’t be necessary in a few months.
The nature of LLMs makes correctly identifying superstition nearly impossible. And the speed with which new models are released makes trying to do so akin to doing physics in a universe where the laws of nature are constantly changing.
You’re an alchemist mixing gunpowder and sacrificing chickens to fire spirits, not an engineer, and for the foreseeable future you have no hope of becoming an engineer.
I’m also highlighting the insanely addictive nature of random rewards.
See this is where I think I fundamentally disagree. It’s more like being a scientist who takes 10 rats and tries to train them to sniff out TB, and once they find one that can do it reliably, they go and do real work with it. Just because the rat is the one doing the sniffing doesn’t mean the scientists aren’t real scientists. And just because we work with a tool that isn’t perfectly reliable doesn’t make us any less engineers as long as we manage and take ownership of the output as we alter and massage it from the machine. Imagine back in the day calling devs that didn’t write in assembly “alchemists” because they didn’t fully understand how a compiler works and just trusted it instead. As long as they’re reading the code, testing the logic, and working towards an outcome they take ownership of, they’re engineers, and so are people who use LLM’s. It’s dangerously close to elitism gatekeeping to say otherwise.
Also I’ve been a software engineer for 15 years so I think I don’t “have no hope of becoming a software engineer”, no personal attacks please.
Ahhh you mean cargo culting!
Yeah it’s definitely cargo culting!
1 reply →