Comment by bojan

1 day ago

> Although, even then, my PM (!) used Claude to boost the efficiency of one of our more complex Postgres queries by up to 5x

Would you be kind to describe what test set-up do you have that a PM felt safe enough to deploy the changed query without the fear they'll break something?

3 comments

bojan

Reply
saghm  21 hours ago

Who said they deployed it themselves? I would expect that they made a PR (probably with some benchmarks), and that there are tests ensuring that the results are the same independent of timing, so then the author of the comment could verify whether it still worked and reproduce the benchmark results and then chose where to approve and merge based on that.

The same techniques that were used to try to prevent regressions before we had LLMs still work for LLM-generated code, provided that you actually use them. If you have an insufficient test suite and a lax policy around who reviews code before merging, you're going to have a bad time even if you only have your seniors writing the code by hand.

bicx  12 hours ago

They built a comparison harness between the old and the new query, and ran it across thousands to benchmark and ensure valid results. And then I reviewed the PR.