Comment by jvanderbot
2 days ago
It's not so contradictory if you peel back a couple layers
1. For junior engineers a 50% increase might be less than a 10% increase for senior engineers. So the % comments are consistent with the pareto argument. Diminishing returns _can produce_ a pareto tradeoff.
2. Saying gains are "concentrated" to junior engineers can also be a function of the work assigned to junior engineers. A senior engineer might get 1000% boost over jr doing the same jr work. The problem is the reverse - the hard things are concentrated around the oldest engineers, and now "hard" can also mean "not easily claude'd"
Yes, I’m a senior engineer at a 3-engineer startup, and there isn’t “junior-level” or “senior-level” work here. It’s all just work. Claude allows me to tear through the kind of gruntwork I’ve been doing ad nauseam for the last 15 years.
If my tasks were things like squeezing 3% more efficiency out of an algorithm, that might be different. Although, even then, my PM (!) used Claude to boost the efficiency of one of our more complex Postgres queries by up to 5x, in ways I honestly would not thought of doing. It was quite humbling. A former engineer here used pgMustard on the same query a couple years ago and claimed it was as fast as it could get.
I can’t speak for the kind of work done in the higher echelons of big tech, but for us, there isn’t much that Claude can’t help with.
Yeah those single points of failure/bottlenecks are prefect focal points for people with floating responsibilities and a good understanding of how to use claude.
The system-sweeping features I still would ask to review, but you can't argue with a DB query just getting faster.
> Although, even then, my PM (!) used Claude to boost the efficiency of one of our more complex Postgres queries by up to 5x
Would you be kind to describe what test set-up do you have that a PM felt safe enough to deploy the changed query without the fear they'll break something?
Who said they deployed it themselves? I would expect that they made a PR (probably with some benchmarks), and that there are tests ensuring that the results are the same independent of timing, so then the author of the comment could verify whether it still worked and reproduce the benchmark results and then chose where to approve and merge based on that.
The same techniques that were used to try to prevent regressions before we had LLMs still work for LLM-generated code, provided that you actually use them. If you have an insufficient test suite and a lax policy around who reviews code before merging, you're going to have a bad time even if you only have your seniors writing the code by hand.
They built a comparison harness between the old and the new query, and ran it across thousands to benchmark and ensure valid results. And then I reviewed the PR.
read the diff and time the query?