Comment by locknitpicker
25 days ago
> I’ve seen AI write a lot of buggy code. I’ve rarely seen AI wrote test cases that expose buggy code.
That's an odd statement to make, particularly with today's models. They can easily pinpoint concurrency problems and memory management issues. But here you are, complaining they write buggy code. What kind of prompting are you throwing at it?
It could be a prompt issue, but I write a lot of concurrent code, and I’ve given it a lot of attempts. I’ve been following model development since word2vec and friends so I think I have a good appreciation of the state of the art and how models understand context.
If there's one theme that's pretty consistent across all the reports I've seen on LLMs for coding, it's that they are both capable of very impressive feats and also capable of screwing up the simplest things.