My experience using LLMs is similar to my experience working with a team of junior developers. And LLMs are valuable in a similar way.
There are many problems where the solution would take me a few hours to derive from scratch myself, but looking at a solution and deciding “this is correct” or “this is incorrect” takes a few minutes or seconds.
So I don’t expect the junior or the LLM to produce a correct result every time, but it’s quick to verify the solution and provide feedback, thus I have saved time to think about more challenging problems where my experience and domain knowledge is more valuable.
I've read other people's code for 25 years for a living. I'm pretty good at it.
So when I get a piece of code or an algorithm, I can read it check if it looks like something I'd allow in a pull review.
We also have this thing called "unit tests", which are pretty good at detecting erroneus code. Some people even write them first and then adjust their code to match.
My experience using LLMs is similar to my experience working with a team of junior developers. And LLMs are valuable in a similar way.
There are many problems where the solution would take me a few hours to derive from scratch myself, but looking at a solution and deciding “this is correct” or “this is incorrect” takes a few minutes or seconds.
So I don’t expect the junior or the LLM to produce a correct result every time, but it’s quick to verify the solution and provide feedback, thus I have saved time to think about more challenging problems where my experience and domain knowledge is more valuable.
Oh, let's check how good you are at spotting security issues... Or plain old bugs.
"It's easy to decide if a solution is correct or not." Yeah right. So you have metrics for it?
The metric is: I run the code and it works?
I've read other people's code for 25 years for a living. I'm pretty good at it.
So when I get a piece of code or an algorithm, I can read it check if it looks like something I'd allow in a pull review.
We also have this thing called "unit tests", which are pretty good at detecting erroneus code. Some people even write them first and then adjust their code to match.
Code review has been an industry standard tool to ensure the correctness of software for decades.
Doesn’t seem to struggle in my experience.