Comment by nickysielicki
18 hours ago
I’m just curious, what would need to happen for you to change your opinion about this? Are you basically of the opinion that it’s not good enough today, never will be good enough in the future, and we should just wind back the clock 3 years and pretend these tools don’t exist?
It feels to me like a lot of this is dogma. If the code is broken or needs more testing, that can be solved. But it’s orthogonal: the LLM can be used to implement the unit testing and fuzz testing that would beat this library into shape, if it’s not already there. It’s not about adding a human touch, it’s about pursuing completeness. And that’s true for all new projects going from zero to one, you have to ask yourself whether the author drove it to completeness or not. That’s always been true.
You want people to hedge their projects with disclaimers that it probably sucks and isn’t production worthy. You want them to fess up to the fact that they cheated, or something. But they’re giving it away for free! You can just not use it if you don’t want to! They owe you nothing, not even a note in the readme. And you don’t deserve more or less hacker points depending on whether you used a tool to generate the code or whether you wrote it by hand, because hacker points don’t exist, because the value of all of this is (and always will be) subjective.
To the extent that the modern tools and models can’t oneshot anything, they’re going to keep improving. And it doesn’t seem to me like there’s any identifiable binary event on the horizon that would make you change your mind about this. You’re just against LLMs, and that’s the way it is, and there’s nothing that anyone can do to change your mind?
I mean this in the nicest way possible: the world is just going to move on without you.
>I’m just curious, what would need to happen for you to change your opinion about this?
Imagine a machine that can calculate using logic circuits and one that uses a lookup table.
LLMs right now is the latter (please don't take literally, It is just an example). You can argue that the look up table is so huge that it works most of the time.
But I (and probably the parent commenter) need it to be the former. And that answers your question.
So it does not matter how huge the lookup table will grow in the future so that it will work more often, it is still a lookup table.
So people are divided into two groups right now. One group that goes by appearance, and one that goes by what the thing actually is fundamentally, despite the appearances.
But logic circuits are look up tables.
Every computation/function can be a look up table!
2 replies →
I think you will get a better response to a slightly different analogy. In genetic programming (and in machine learning), we have a concept of "overfitting". Overfitting can be understood as a program memorizing too much of its test/training data (i.e. so it is acting more like an oracle than a computation). This, intuitively, becomes less of a problem the greater the training-dataset becomes, but the problem will always be there. Noticing the problem is like noticing the invisible wall at the edge of the game-world.
The most insightful thing about LLMs, is just how _useful_ overfitting can be in practice, when applied to the entire internet. In some sense, stack-overflow-driven-development (which was widespread throughout the industry since at least 2012), was an indication that much of a programmer's job was finding specific solutions to recurring problems, that never seem to get permanently fixed (mostly for reasons of culture, conformity, and churn in the ranks).
The more I see the LLM-ification of software unfold (essentially an attempted controlled demolition of our industry and our culture), the more I think about Arthur Whitney (inventor of the K language and others). In this interview[1], he said two interesting things: (1) he likened programming to poetry, and (2) he said that he designed his languages to not have libraries, and everybody builds from the 50 basic operators that come with the language, resulting in very short programs (in terms of both source code size and compiled/runtime code size).
I wonder if our tendency to depend on libraries of functions, counterintuitively results in more source code (and more compiled/runtime code) in the long run -- similarly to how using LLMs for coding tends to be very verbose as well. In principle, libraries are collections of composable domain-verbs that should allow a programmer to solve domain-problems, and yet, it rarely feels that way. I have ripped out general libraries, and replaced them with custom subroutines more times than I can count, because I usually need a subset of functionality, and I need it to be correct (many libraries are complex and buggy because they have some edge-cases [for example, I once used an AVL library that would sometimes walk the tree in reverse instead of from least to greatest -- unfortunately, the ordering mattered, and I wrote a simpler bespoke implementation]).
Arguably, a buggy program or a buggy library or a buggy function, is just an overfit program, or library, or function (it is overfit to the mental-model of the problem-space in the library writer's mind). These overfit libraries, which are often used as blackboxes by someone rushing to meet a deadline, often result in programs that are themselves overfit to the buggy library, creating _less_ modularity instead of more. _Creating_ an abstraction is practically free, but maintaining it and (most disappointingly) _using_ it has real, often permanent long term costs. I have rarely been able to get two computers, that were meant to share data with NFS, to do so reliably, if they were not running the same exact OS (because the NFS client and server of each OS are bug-compatible, are overfit to each other).
In fact the rise of VMWare, and the big cloud companies, and containerization and virtualization technologies is, conceivably, caused by this very tendency to write software that is overfit to other software (the operating system, the standard library [on some OSes emacs has to be forced to link to glibc, because using any other memory allocator causes it to SEGFAULT, and don't get me started on how no two browser-canvases return the same output in different browser _nor_ on the same browser in a different OS]). (Maybe, just as debt keeps the economy from collapsing, technical debt is the only thing that keeps Silicon Valley from collapsing.)
In some ways, coding-LLMs exaggerate this tendency towards overfitting in comical ways, like fun-house mirrors. And now, a single individual, with nothing but a dream, can create technical debt at the same rate as a thousand employee software company could a decade ago. What a time to be alive.
[1]: https://queue.acm.org/detail.cfm?id=1531242
>he likened programming to poetry
This I can definitely relate..
I don't fully understand the rest, but I ll give it some thought.
This might be true, but we can continue to try and require the communities we have been part of for years to act a certain way regarding disclosures.
If the community majority changes it mind then so be it. But the fight will continue for quite some time until that is decided.
There never was a cohesive generic open source community. There are no meaningful group norms. This was and always will be a fiction.
I’m tempted to just start putting co-authored-by: Claude in every commit I make, even the ones that I write by hand, just to intentionally alienate people like you.
The best guardrails are linters, autoformatters, type checkers, static analyzers, fuzzers, pre-commit rules, unit tests and coverage requirements, microbenchmarks, etc. If you genuinely care about open source code quality, you should be investing in improving these tools and deploying them in the projects you rely on. If the LLMs are truly writing bad or broken code, it will show up here clearly.
But if you can’t rephrase your criticism of a patch in terms of things flagged by tools like those, and you’re not claiming there’s something architecturally wrong with the way it was designed, you don’t have a criticism at all. You’re just whining.
> There never was a cohesive generic open source community. There are no meaningful group norms. This was and always will be a fiction.
It's always been a bit splintered, but it was generally composed of 95%+ of people that know how to program. That is no longer the case in any sense.
> I’m tempted to just start putting co-authored-by: Claude in every commit I make, even the ones that I write by hand, just to intentionally alienate people like you.
I mean it sounds like you are already using claude for everything so this is probably a bit of a noop lol.
> But if you can’t rephrase your criticism of a patch in terms of things flagged by tools like those, and you’re not claiming there’s something architecturally wrong with the way it was designed, you don’t have a criticism at all. You’re just whining.
No, because doing that requires MORE rigor and work than what an LLM driven project had put into it. That difference in effort/work is not tenable, its shallow work being shown, its shallow criticisms thrown at it.
All sense of depth and integrity is gone and killed.
2 replies →
I see this as the same argument as saying GMO label not needed, no need to mention artificial flavours in food, etc.
I mean this in the nicest way possible: the world is just going to insist that AI generated output is marked clearly as AI produced output.
Not sure whether giving a LICENSE even makes sense.
I tried to control LLM output quality by different means, including fuzzing. Had several cases when LLM "cheated" on that too. So, I have my own shades and grades of being sure the code is not BS.
Well, that’s obviously bad.
But once you told it to stop cheating, did it eventually figure it out? I mean, correctly implementing fuzzer support for a project is entirely within the wheelhouse of current models. It’s not rocket science.
You’ve gotta read the code. It doesn’t matter how it got there but if you don’t fully understand it (which implies reading it) don’t get mad when you try to push slop on people. It’s the equivalent of asking an LLM to write an email for somebody else to read that you didn’t read yourself. It’s basic human trust - of course people get annoyed with you. You’re untrustworthy.