Comment by besterman23
8 hours ago
That’s something that I have been thinking about for a while now. Even prior to LLMs, many large companies have essentially been iterating on the same half a dozen ideas over and over again and there doesn’t seem to be much new coming out even post LLMs.
The most recent new things that LLMs have unlocked (regarding software dev) for us is reverse engineering and combining old games, unearthing decade old vulnerabilities, and helping root and debloat consumer hardware. Which are all cool and useful, but none of it being a “next great thing”.
Perhaps LLMs are really doing is highlighting how the software industry was already nearing its economic maturity and this just shifted the timeline from 2040 back to 2027-2030.
Don’t forget we don’t live in a pacifist utopia. Companies don’t work on the same ideas just because they think they can make it better.
Companies work on the same ideas because they want to own IP, data etc. and then charge money.
The other way around they work on the same ideas not to be charged by others and control their business.
“software industry” is not some big happy hippie family. Patent wars, browser wars, OS wars - well we get a lot of open source for free but that’s not the norm.
I have yet to see LLMs do well “autonomously” on anything that doesn’t fall into any of these 3 buckets:
1. well specified
2. well tested (which is a form of spec)
3. represented in the training set (i.e. not built on a corpus of human developed examples)
For example, a lot of the compiler porting efforts fall into all 3 buckets: compilers are well specified (there is a set of valid inputs and an expected behavior), they are typically well tested, and as a bonus, in a way that is implementation agnostic thanks to the clean interface (think: integration vs unit tests), and are very well represented in the training set.
You can "solve" 1 and 2 by N-version programming. Have the same code implemented twice, or more times, you get free error testing by fuzzing or running replicas in shadow mode. In order to make replicas maximally different choose a different language, framework, agent harness and LLM.
There remain some nitty details related to tracking randomness and choosing what interfaces to log. But even having your code testable by replicas would help organize it better and expose different assumptions made by the language or library stack.
When you get different outputs from replicas it can be 1. one of them has a bug, or 2. the spec is unclear -> fix the bug or the spec. So this method catches both sides. It also might be faster than traditional code based testing, you can build replicas in parallel. A replica is an open ended test, unlike coded tests it can be tried on open ended input values.
The core idea is that it is easier to implement twice than to imagine bugs. We already solved implementation, but bug finding means running code in your head, it touches the halting problem issues - can we really imagine behavior from code? I don't think we can do it well. That is why we should just make replicas instead, tokens are getting cheaper and cheaper by the month.
and yet some OSS (projects not really known for 1 and 2) multiplied change velocity the last year, implementing features via what is essentially vibecoding.
If you look at that, this approach is producing a pile of slop, where one LLM implements a feature, two weeks later another LLM spots an edge case and without much human review just crams in the code to fix that. Even at 1-2 months of that, those repos have significant changes and the commit messages get weird. I wonder what happens if the LLM won't spit out a fix anymore?
Will the human maintainers who started this process because noone thanked them and noone wanted to work with them on their bright new ideas just reset to the last commit they understand? Will they drop the project? The sunk cost for them is enormous. The time-effort required to fix the mess is gargantuan already. To me it increasingly seems that after the LLM industry robbed most humans of their collective labour output it will now rob them of labour aka time as well. Some people (like you and me) seem to notice that but most just go on and essentially spend away their life on machines now. I guess the problem is that most of these people have been severely cut off from using their own agency.
I had an LLM tell me that I mattered once, which I know for objective fact is not represented in the training data.
it is and you do matter
Honestly I think you're on to something, as by far the most novel and interesting use cases I've seen for LLMs in general are mostly around reverse engineering things, especially if you can hit it with Ghidra, but honestly anything that can feed some sort of data and control back to a model.
> the software industry was already nearing its economic maturity
What I'm sensing is something I might describe as the Oligarchy reaching the point of significant societal pushback, like the labor rights movements that ended 80 work weeks, unsafe conditions, and child labor in factories (i.e. The Jungle).
My hope is that we can use this moment in human history to break the user tracking and targeted advertising industry. It's been so harmful to humans, but in doing so, we will likely break a major revenue source that supports our well above average salaries. I still think it should be done.
What indications do you see of significant and effective societal pushback?
I would love to believe this but I don't see it yet. I see a lot of anger in some circles, but I also see the machine marching on as normal.
What would be funny with LLMs, having been purposefully trained to be the best software creating things on earth, wind up inadvertently making software virtually valueless (because it can be made and copied so easily) and undermining the value of the companies that proliferated their use in the first place.
> What would be funny with LLMs, having been purposefully trained to be the best software creating things on earth, wind up inadvertently making software virtually valueless
This is the world the accelerationists are trying to create, is it not? Fully automated luxury space communism through AI solving all earthly problems.
We're just unlucky that software is one of the first moats to fall (alongside the humanities), because it doesn't require developing novel ways for the AI to interact with the real world...
We'll have to separate the hardware providers from the software providers, which I'm growing more fond of. The US Ai stack is largely a vertically integrated cartel.