Comment by 20k
2 hours ago
>If agentic development actually worked the way any of them say it does
I think its fascinating just how much of a gap there is between what's being claimed, and the verifiable observable data of the open source world. Major open source projects are by and large starting to ban LLMs now, because the contributions made by LLM users have been universally terrible and unhelpful. There doesn't appear to be a single major project that's found generating code to lead to major productivity speedups, and the consensus appears to be that its just lead to a lot of crappy contributions that are harder to spot immediately as being obvious crap
I regularly see people claim that they are now 10x more productive with LLM code generation, and I just wonder where all the code is. Is it somehow true that these gains are only being realised in proprietary projects, and not a single one of them has put even a small fraction of their new found engineering powers into eg Godot? Why do only the poor quality LLM code generation users make PRs to open source projects, and never the engineers that know how to really use it correctly?
If you look in the open source major project space, you can find almost no evidence that AI code generation exists at all. Go browse your favourite critical tool and look for AI generated PRs that have landed in the codebase, its probably a tiny handful of them in comparison to the human written PRs prior to an LLM ban. It turns out that once you have a verifiable, open quality review bar, for some reason almost no LLM commits really meet the level of quality necessary
I strongly suspect that what we're seeing is that much of the tech code-writing economy had already become completely performative prior to AI turning up. It no longer matters in the current age if your code is good, or works, because your job is to give the illusion of product development while the stock market price gets pumped, until you all cash out your share value, get bought, or hop jobs in 2 years. For many companies it literally does not matter if you produce anything that generates value (or works), because the illusion of progress is all that matters. AI is absolutely incredible at creating the illusion of progress, because it looks a whole lot like real code, it just appears to have failed the bar of making actual projects that work. If that was never the goal in the first place, it probably really is a 10x productivity boost
@antirez is a very prominent open source contributor that gets lot of shit done with LLMs. Mitchel Hashimoto is also open about using LLMs to speed up his work. There are some caveats attached: neither of them is doing crazy loops or graphs producing thousands of lines of code, they are both amazing software engineers and they know what they are doing.
> I think its fascinating just how much of a gap there is between what's being claimed, and the verifiable observable data of the open source world.
According to this list of top Github repos by stars[1], of the top 10, nine of them are informational and one is OpenClaw, which certainly has plenty of AI generated code.
The next 10 include react which has a CLAUDE.md and a .claude, and lots of landed PRs that look like AI. And linux, which, we all read Linus' stance. He sees the value. And superpowers, ECC, and hermes-agent. All of which are AI stuff.
I think it would be reasonable to ask which major open source projects aren't using AI assistance?
> Major open source projects are by and large starting to ban LLMs now, because the contributions made by LLM users have been universally terrible and unhelpful.
That is not why. Or at least not the majority of it. The biggest part is that maintainers just can't keep up. They don't have time to do in depth reviews for the number of PRs coming in to find out if it: works, does what it says it does, meets a need, has docs, is well written, meets criteria, etc. That does not mean all those PRs are trash. Some might be trash, some might be gold. Just like human generated ones.
What it comes down to is vibes. Anti-AI folk don't like AI for reasons (that are valid) and they don't want to find any value in it. So they don't. But lots of people are. Lots of really cool, interesting, clever, new software is being written and getting used and building communities but people see CLAUDE.md and go "slop, no thanks" and won't even give it a try.
[1]: https://github.com/EvanLi/Github-Ranking/blob/master/Top100/...
> Why do only the poor quality LLM code generation users make PRs to open source projects, and never the engineers that know how to really use it correctly?
It could be that the engineers who are extremely productive with LLMs are landing PRs that look indistinguishable from good, hand-written PRs.
If so, where are all the new features in the open-source projects I use? Why hasn't GIMP replicated Photoshop? Why hasn't CUDA been fully reverse-engineered as an open source toolchain? These are unreasonable expectations, but only in response to unreasonable claims of productivity. What before took ten years should now only take one, right?
It seems likely that the gains from generating tons of code are being offset by the debt incurred to understanding what you're doing. We see lots of greenfield projects one-shotted with GPT or GLM or whatnot, but very little on the side of projects with long-term maintenance goals. This is telling, to me, that the _effective_ gains are much lower than perceived (it's lots of fun to see the thing crank out code at breakneck pace, probably contributing to this). Still quite nice, and very useful, but not a totally new paradigm.
> Why hasn't GIMP replicated Photoshop?
My guess is that open source folks overindex on being anti-AI.
1 reply →
Yeah, why hasn't AMD added every single CUDA feature lol
1 reply →
If that were true, we'd expect to see massively accelerated velocity of open source projects by these engineers. They should be creating new open source projects at a truly astounding rate, with new tooling springing up every day that dwarfs the existing open source space as their productivity completely eclipses traditional development
Instead, software is plodding along exactly the same as it did prior to LLM code generation, and there's no evidence of superprogrammers making superprojects in 1/10th of the time. With a 10x productivity gain, what used to take a year should take a month
Github's Octoverse report kind of gives you that evidence, no? Pull requests landed in 2025 were up 30% over 2024. This year it's going to be much higher.
1 reply →
LLM code is obvious to spot. If an LLM designed a screwdriver set it would make a screwdriver for each head instead of making a replaceable head.
LLMs are superhuman at short term coding such as debugging and writing tests though and you're missing out by not using them there.
So... basically an autocomplete to help you type faster and spot check if you did something unusual in the parts you wrote by hand?
> LLM code is obvious to spot. If an LLM designed a screwdriver set it would make a screwdriver for each head instead of making a replaceable head.
Commercial and industrial electricians are LLMs now?
> I regularly see people claim that they are now 10x more productive with LLM code generation, and I just wonder where all the code is.
Perhaps the biggest issue hasn't been the speed of development all along but people working on wrong things, hence why progress isn't as evident. LLM helping to build something that isn't needed faster isn't going to productivity more visible.
There's no reason that some people wouldn't be using their new found 10x superpowers on open source projects though
Major open source projects are by and large starting to ban LLMs now, because the contributions made by LLM users have been universally terrible and unhelpful.
There are two incentives for contributing to open source. The first is to make the app better (add a feature, improve the code, fix a bug, etc.) It's possible that LLMs don't meet the bar but if a human has put the effort in it's not always obvious that it's AI. More likely LLM code is accepted when it's good and rejected when it's bad. The system works.
However, the second incentive is that open source contributions are seen as a 'ahortcut' to making a name for yourself. Being a contributor on a big project goes on resumes and GitHub profiles. Often people who use AI for that don't review the code or even check it does what they say it does. That slop needs to be banned. The downside is losing the good contributions, but it's still a net win.
We have a greenfield project at our company, yes proprietary, now taking us months where previously it'd have taken weeks for even a single feature. We definitely see the (whatever)x performance boost with our own eyes.
Its always been possible to trade long term productivity for short term gains with technical debt. This is why the bar I'm interested in is long term projects, which have proven to have long term success, instead of a small disposable project where the code quality doesn't matter
> now taking us months where it'd have taken weeks...
So 0<whatever<1
Nah, the GP's claim is weeks for each feature, versus a few months in total. On a greenfield project.
Weeks for each feature in a greenfield project... Yeah, LLMs make unworking code way faster than that.
Nah this take is wrong. I used Claude code with my custom skill and I wrote a more performant scheduler than the default Linux one in Rust.
It's not just productivity, it's life changing.
Sure, it just seems a little odd that no LLMgineer ever contributes their incredible more performant scheduler back though right? After all if you can do it with claude, anyone can, all it'd take is to ask claude to rewrite it. Linux accepts LLM generated PRs, all the code has to do is meet the review bar and one of the most critical pieces of software engineering on the planet gets better for everyone
Why does it seem odd when people who do attempt to contribute back are attacked?
AI lowers the bar to making submissions which permits flawed subissions to be thoughtlessly submitted by people who don't really understand what they are doing, technically or culturally.
It does, however, enable people who put in the effort to work on something significant. Those people are fewer in number, but exist. They do get to see the animosity that unwitting novices receive.
I don't think it would be surprising for those people to opt out of engaging with a toxic environment.
3 replies →
I would submit it but the pr was too long to fit on GitHub
3 replies →
I'm choosing to interpret this as very dry sarcasm.
With the "It's not X, it's Y" at the end I sure hope that it's the case.
[dead]
> Why do only the poor quality LLM code generation users make PRs to open source projects, and never the engineers that know how to really use it correctly?
Selection bias? OSS has a thing proprietary projects don't - an endless cohort of opportunists trying to wedge in a "contribution" for personal gain, be it a Hacktober t-shirt or resume boost or an occasional vulnerability (with the resume boosting being by far most likely). The good LLM-based contributions, you probably won't recognize as AI-assisted unless the author explicitly decided to label them, and if they're really good and use LLMs responsibly, they probably don't even have any ethical obligation to label LLM involvement, much less any benefit.
OSS involvement in general stopped being an indicator of skill once Github activity became a factor for job applications.
Edit:
There's also a second factor: many people (myself included) use LLMs to code one-off personal tools with no intent of them having more than one user, ever. Get an itch, have an LLM scratch it for you, carry on with your life. You'll see little trace of it, because it's hard to distinguish a single-user program that's good enough for specific purpose from a pile of autogenerated slop that probably doesn't work, and neither of these will show up as PRs to big OSS repos anyway.
The the idea that engineers are covertly sneaking huge quantities of LLM generated code into these projects doesn't pass the smell test, as there's lots of evidence that this isn't true. It'd be incredibly obvious if new contributors were turning up en masse and contributing absolutely incredible amounts of high quality code, because everyone would be crowing about it
Instead, if you check out something like ImGui, we can see that its largely just Omar as usual with a bunch of PRs. Nothing much seems to have changed, feature development carries on at the same pace as it did previously. This is replicated across nearly every single major long term open source project I can find
The only reason developers now feel the need to hide their LLM usage is because the LLM contributions have all been shockingly bad, so you're reversing cause and effect. If LLM contributions had been incredible, they wouldn't be frowned upon socially now. Surely someone would blow the lid on this huge covert conspiracy about how incredible their contributions to.. SFML have been?
This effect also wouldn't prevent LLMgineers from creating their own open source projects that should be absolutely outstripping the existing open source space. After all how old is ImGui? With a 10x productivity gain, it should take a year of development to easily match its features if those gains are truly real now
>if they're really good and use LLMs responsibly, they probably don't even have any ethical obligation to label LLM involvement
The copyright issues means that non disclosure is inherently unethical
> The only reason developers now feel the need to hide their LLM usage is because the LLM contributions have all been shockingly bad, so you're reversing cause and effect. If LLM contributions had been incredible, they wouldn't be frowned upon socially now
You're dismissing scale of things. OSS communities were having high-profile discussions about gatekeeping and discrimination way before LLMs were a thing, because already at that point the amount of human generated slop was becoming untenable. And yet, no one argued that this must be evidence that all human contributions are bad and humanity is a bubble waiting to pop.
> ImGui
Weird choice of an example. ImGui is common infrastructure. Who in their right mind would want to mess with that? Especially when building something to solve some end-user problem (whether N=1 or N=1 billion), ImGui is a component you include, not one you rewrite.
There's hardly any reason anyone has to contribute to that project, with or without LLMs.
Or maybe open-source development is just poorly compatible with AI workflow? Today, projects may need a community, an issue tracker, but pull requests are becoming less important.
I suspect the ban of AI in established projects to be a very complex decision. Even if core developers would like to use AI, they don't want to review all of the AI-generated code from the larger community. So the only consistent way to preserve sanity is to declare that nobody would use it. Good luck enforcing it though!
Why would the PR format be bad for LLMs? Its just code review + merging in branches, which is what every company should be doing anyway
Nothing's stopping core developers from adopting LLM generated code for themselves, while banning it for external contributors
> Nothing's stopping core developers from adopting LLM generated code for themselves, while banning it for external contributors
And if they're doing that, why would they tell you, or anyone?
And if they're doing that responsibly - collaborating with AI, and reviewing the code - they don't even have an ethical reason to tell about AI involvement, any more than telling about the StackOverflow answers or blog posts they read before coming up with some implementation.
1 reply →