Comment by simonw
2 hours ago
Here are a few examples you might find credible:
- pola-rs/polars: https://github.com/pola-rs/polars/pull/26823 - ~2.68x median speedup of primitive-to-boolean casting credited to Claude Opus 4.6
- pydantic/monty: https://github.com/pydantic/monty/pull/643 - ~53x speedup (488ms down to 9.2ms) of bytes substring search generated with Claude Code
- numpy/numpy: https://github.com/numpy/numpy/pull/31573 - ~21x speedup of Python datetime → datetime64 conversion, with Claude Code used for profiling and implementing the performance improvements
- apache/datafusion: https://github.com/apache/datafusion/pull/21182 - up to 49x speedup of LIMIT queries by eliminating unnecessary sorts and pushing limits into file scans, generated with Claude Code
- apache/datafusion: https://github.com/apache/datafusion/pull/21651 - ~4.39x speedup of ClickBench Q6 by resolving MIN/MAX directly from Parquet metadata instead of scanning columns, generated with Claude Code
- pola-rs/polars: https://github.com/pola-rs/polars/pull/27958 - ~3.2x speedup of Int8 Series sum, developed with assistance from Claude Fable and Claude Opus 4.8
- numpy/numpy: https://github.com/numpy/numpy/pull/31274 - up to ~1.44x speedup of common small NumPy reductions, with the fast path written by Claude Code and manually refined
That's a fair use case for them though. LLMs are superhuman at short term performance engineering/debugging/testing. But even that comes with the caveat that these commits come from talented coders using LLMs as a grad student. I think that OP is against the claim that "agent in a loop beats a talented human at long term coding tasks".
Because if it was true then open source projects such as GIMP could basically be as feature heavy as Photoshop overnight.
I, too, will reject the idea that "agent in a loop beats a talented human at long term coding tasks".
LLMs amplify existing expertise. Give them to experts and you can get fantastic results. Give them to amateurs and you might get the occasional impressive demo, but you're not going to get anything that a responsible software team would commit to maintaining in the long term.