Comment by jgraham
2 months ago
> it's foolish to fight the future
And yet, the premise of the question assumes that it's possible in this case.
Historically having produced a piece of software to accomplish some non-trivial task implied weeks, months, or more of developing expertise and painstakingly converting that expertise into a formulation of the problem precise enough to run on a computer.
One could reasonably assume that any reasonable-looking submission was in fact the result of someone putting in the time to refine their understanding of the problem, and express it in code. By discussing the project one could reasonably hope to learn more about their understanding of the problem domain, or about the choices they made when reifying that understanding into an artifact useful for computation.
Now that no longer appears to be the case.
Which isn't to say there's no longer any skill involved in producing well engineered software that continues to function over time. Or indeed that there aren't classes of software that require interesting novel approaches that AI tooling can't generate. But now anyone with an idea, some high level understanding of the domain, and a few hundred dollars a month to spend, can write out a plan can ask an AI provider to generate them software to implement that plan. That software may or may not be good, but determining that requires a significant investment of time.
That change fundamentally changes the dynamics of "Show HN" (and probably much else besides).
It's essentially the same problem that art forums had with AI-generated work. Except they have an advantage: people generally agree that there's some value to art being artisan; the skill and effort that went into producing it are — in most cases — part of the reason people enjoy consuming it. That makes it rather easy to at least develop a policy to exclude AI, even if it's hard to implement in practice.
But the most common position here is that the value of software is what it does. Whilst people might intellectually prefer 100 lines of elegant lisp to 10,000 lines of spaghetti PHP to solve a problem, the majority view here is that if the latter provides more economic value — e.g. as the basis of a successful business — then it's better.
So now the cost of verifying things for interestingness is higher than the cost of generating plausibly-interesting things, and you can't even have a blanket policy that tries to enforce a minimum level of effort on the submitter.
To engage with the original question: if one was serious about extracting the human understanding from the generated code, one would probably take a leaf from the standards world where the important artifact is a specification that allows multiple parties to generate unique, but functionally equivalent, implementations of an idea. In the LLM case, that would presumably be a plan detailed enough to reliably one-shot an implementation across several models.
However I can't see any incentive structure that might cause that to become a common practice.
>a plan detailed enough to reliably one-shot an implementation across several models.
What. Why should this be an output? Why if I make a project should I be responsible for also making this, an entirely different and much more difficult and potentially impossible project? If I come and show you a project that require required thousands of sessions to make I also have to show you how to one shot it in multiple models? Does that even make sense?
To be clear: I don't think it will happen.
But the point of comparison is something like the HTML specification. That's supposed to be a document that is detailed enough about how to create an implementation that multiple different groups can produce compatible implementations without having any actual code in common.
In practice it still doesn't quite work: the specification has to be supplemented with testsuites that all implementations use, and even then there often needs to be a feedback loop where new implementations find new ambiguities or errors, and the specification needs to be updated. Plus implementors often "cheat" and examine each other's behaviour or even code, rather than just using the specification.
Nevertheless it's perhaps the closest thing I'm familiar with to an existing practice where the plan is considered canonical, and therefore worth thinking about as a model for what "code as implementation detail" would entail in other situations.
I think the looping part is what stops this from being a practical solution. If we imagine that the actual code required some iteration in order to put down, I don’t know that we could say there is a one shot equivalent without testing that. Sometimes there may not even be an equivalent.
It’s possible that the solution to code being implementation detail is to be less precious about it and not more. I don’t really have an answer here and I don’t think anyone does because it’s all very new and it is hard to manage.
There’s also a pretty normal way in which this is going to diverge and perhaps already has. Developers are building local bespoke skills just like they used to develop and still do local bespoke code to make their work more efficient. They may be able to do something that you or I cannot using the same models—-there’s no way to homologize their output. It would be like asking someone to commit their dot files alongside the project output. Regardless of whether or not it was the right thing to do no one would do it.