Comment by sho_hn
5 days ago
Also ironic: When the same professionals advocating "don't look at the code anymore" and "it's just the next level of abstraction" respond with outrage to a journalist giving them an unchecked article.
Read through the comments here and mentally replace "journalist" with "developer" and wonder about the standards and expectations in play.
Food for thought on whether the users who rely on our software might feel similarly.
There's many places to take this line of thinking to, e.g. one argument would be "well, we pay journalists precisely because we expect them to check" or "in engineering we have test-suites and can test deterministically", but I'm not sure if any of them hold up. The "the market pays for the checking" might also be true for developers reviewing AI code at some point, and those test-suites increasingly get vibed and only checked empirically, too.
Super interesting to compare.
- There’s a difference. Users don’t see code, only its output. Writing is “the output”.
- A rough equivalent here would be Windows shipping an update that bricks your PC or one of its basic features, which draws plenty of outrage. In both cases, the vendor shipped a critical flaw to production: factual correctness is crucial in journalism, and a quote is one of the worst things to get factually incorrect because it’s so unambiguous (inexcusable) and misrepresents who’s quoted (personal).
I’m 100% ok with journalists using AI as long as their articles are good, which at minimum requires factual correctness and not vacuous. Likewise, I’m 100% ok with developers using AI as long as their programs are good, which at minimum requires decent UX and no major bugs.
> - There’s a difference. Users don’t see code, only its output. Writing is “the output”.
So how is the "output" checked then? Part of the assumption of the necessity of code review in the first place is that we can't actually empirically test everything we need to. If the software will programmatically delete the entire database next Wednesday, there is no way to test for that in advance. You would have to see it in the code.
Tbf I'm fine with it only one way around; if a journalist has tonnes of notes and data on a subject and wants help to condense those down into an article, assistance with prioritising which bits of information to present to the reader then totally fine.
If a journalist has little information and uses an llm to make "something from nothing" that's when I take issue because like, what's the point?
Same thing as when I see managers dumping giant "Let's go team!!! 11" messages splattered with AI emoji diarrhea like sprinkles on brown frosting. I ain't reading that shit; could've been a one liner.
Another good use of an LLM is to find primary sources.
Even an (unreliable) LLM overview can be useful, as long as you check all facts with real sources, because it can give the framing necessary to understand the subject. For example, asking an LLM to explain some terminology that a source is using.
Excellent observation. I get so frustrated every time I hear the "we have test-suites and can test deterministically" argument. Have we learned absolutely nothing from the last 40 years of computer science? Testing does not prove the absence of bugs.
Don't worry, the LLM also makes the tests. /s
Trading pesky things like accuracy and employees for compute/shell games; money-concentrating machine go brrrrrt
I look forward to a day when the internet is so uniformly fraudulent that we can set it aside and return to the physical plane.
I don't know if I look forward to it, myself, but yeah: I can imagine a future where in person interactions become preferred again because at least you trust the other person is human. Until that also stops being true, I guess.
There's a fracking cylon on Discovery!
Well, I can tell you I've been reading a lot more books now. Ones published before the 2020s, or if recent, written by authors who were well established before then.
Physical books are amazing technology.
I've started suspecting I'm looking back of the day where that happened.
Because nobody ever lied in print media or in person?
What we are seeing is the consequence of a formerly high trust society collapsing into a low trust one. There is no place to hide from that. The Internet is made of the same stuff as print media and in person. It’s made of people.
The internet didn’t cause this. It just reflects it.
The LLMs are made of people too inasmuch as that’s where they get their training data and prompts.
> When the same professionals advocating "don't look at the code anymore" and "it's just the next level of abstraction" respond with outrage to a journalist giving them an unchecked article.
I would expect there is literally zero overlap between the "professionals"[1] who say "don't look at the code" and the ones criticising the "journalists"[2]. The former group tend to be maximalists and would likely cheer on the usage of LLMs to replace the work of the latter group, consequences be damned.
[1] The people that say this are not professional software developers, by the way. I still have not seen a single case of any vibe coder who makes useful software suitable for deployment at scale. If they make money, it is by grifting and acting as an "AI influencer", for instance Yegge shilling his memecoin for hundreds of thousands of dollars before it was rugpulled.
[2] Somebody who prompts an LLM to produce an article and does not even so much as fact-check the quotations it produces can clearly not be described as a journalist, either.
> When the same professionals advocating "don't look at the code anymore" and "it's just the next level of abstraction" respond with outrage to a journalist giving them an unchecked article.
I doubt, by and large, that it's the same people. Just as this LLM misquoting is journalistic malpractice, "don't look at the code anymore" is engineering malpractice.
While I don't subscribe to the idea that you shouldn't look at the code - it's a lot more plausible for devs because you do actually have ways to validate the code without looking at it.
E.g you technically don't need to look at the code if it's frontend code and part of the product is a e2e test which produces a video of the correct/full behavior via playwright or similar.
Same with backend implementations which have instrumentation which expose enough tracing information to determine if the expected modules were encountered etc
I wouldn't want to work with coworkers which actually think that's a good idea though
If you tried this shit in a real engineering principle, you'd end up either homeless or in prison in very short order.
You might notice that these real engineering jobs also don't have a way to verify the product via tests like that though, which was my point.
And that's ignoring that your statement technically isn't even true, because the engineers actually working in such fields are very few (i.e. designing bridges, airplanes etc).
The majority of them design products where safety isn't nearly as high stakes as that... And they frequently do overspec (wasting money) or underspec (increasing wastage) to boot.
This point has been severely overstated on HN, honestly.
Sorry, but had to get that off my chest.
5 replies →
I’ve been saying the same kind of thing (and I have been far from alone), for years, about dependaholism.
Nothing new here, in software. What is new, is that AI is allowing dependency hell to be experienced by many other vocations.
I haven't seen a single person advocate not looking at the code.
I'm sure that person exists but they're not representative of HN as a whole.
> the same professionals
Same forum, not necessary same people.
So much projection these days in so many areas of life.