Comment by jsenn
3 days ago
Is this parlour trick so different from useful tasks like “implement this feature while following the naming conventions of my project”?
3 days ago
Is this parlour trick so different from useful tasks like “implement this feature while following the naming conventions of my project”?
The difference is that in a software project you can throw more than one instance of the model at the code. If you tell it to follow your naming conventions and it fails to do so, that can be picked up by an instance of the same LLM that's running checks before you commit anything. Even though it's the same model it'll usually detect stuff like that. You can even have it do multiple passes.
The way most people are coding with AI today is like Baby's First AI™ compared to how we'll all be using LLMs for coding in the future. Soon that "double check everything" step will be built in to the coding agents and you'll have configuration options for how many passes you want it to perform (speed VS accuracy tradeoff).
From the model's perspective it's completely different. LLMs have no concept of what a letter is due to the way they're trained.
If your code base looks like five random animal names then I guess not.