Comment by Springtime

- There’s a difference. Users don’t see code, only its output. Writing is “the output”.

- A rough equivalent here would be Windows shipping an update that bricks your PC or one of its basic features, which draws plenty of outrage. In both cases, the vendor shipped a critical flaw to production: factual correctness is crucial in journalism, and a quote is one of the worst things to get factually incorrect because it’s so unambiguous (inexcusable) and misrepresents who’s quoted (personal).

I’m 100% ok with journalists using AI as long as their articles are good, which at minimum requires factual correctness and not vacuous. Likewise, I’m 100% ok with developers using AI as long as their programs are good, which at minimum requires decent UX and no major bugs.

Excellent observation. I get so frustrated every time I hear the "we have test-suites and can test deterministically" argument. Have we learned absolutely nothing from the last 40 years of computer science? Testing does not prove the absence of bugs.

I look forward to a day when the internet is so uniformly fraudulent that we can set it aside and return to the physical plane.

> When the same professionals advocating "don't look at the code anymore" and "it's just the next level of abstraction" respond with outrage to a journalist giving them an unchecked article.

I would expect there is literally zero overlap between the "professionals"[1] who say "don't look at the code" and the ones criticising the "journalists"[2]. The former group tend to be maximalists and would likely cheer on the usage of LLMs to replace the work of the latter group, consequences be damned.

[1] The people that say this are not professional software developers, by the way. I still have not seen a single case of any vibe coder who makes useful software suitable for deployment at scale. If they make money, it is by grifting and acting as an "AI influencer", for instance Yegge shilling his memecoin for hundreds of thousands of dollars before it was rugpulled.

[2] Somebody who prompts an LLM to produce an article and does not even so much as fact-check the quotations it produces can clearly not be described as a journalist, either.

> When the same professionals advocating "don't look at the code anymore" and "it's just the next level of abstraction" respond with outrage to a journalist giving them an unchecked article.

I doubt, by and large, that it's the same people. Just as this LLM misquoting is journalistic malpractice, "don't look at the code anymore" is engineering malpractice.

While I don't subscribe to the idea that you shouldn't look at the code - it's a lot more plausible for devs because you do actually have ways to validate the code without looking at it.

E.g you technically don't need to look at the code if it's frontend code and part of the product is a e2e test which produces a video of the correct/full behavior via playwright or similar.

Same with backend implementations which have instrumentation which expose enough tracing information to determine if the expected modules were encountered etc

I wouldn't want to work with coworkers which actually think that's a good idea though

I’ve been saying the same kind of thing (and I have been far from alone), for years, about dependaholism.

Nothing new here, in software. What is new, is that AI is allowing dependency hell to be experienced by many other vocations.

I haven't seen a single person advocate not looking at the code.

I'm sure that person exists but they're not representative of HN as a whole.

> the same professionals

Same forum, not necessary same people.

So much projection these days in so many areas of life.

This is just one of the reasons archiving is so important in the digital era; it's key to keeping people honest.

I read the forum thread, and most people seem to be critical of ars. One person said scott is a bot, but this read to me as a joke about the situation

The comment calling him a bot is sarcasm.

It’s funny they say the article “may have” run afoul of their journalistic standards. May have is carrying a lot of weight there.

Just like in the original thread that was wiped (https://arstechnica.com/civis/threads/um-what-happened-to-th...

TBF even journalists who interview people for real and take notes routinely quite them saying things they didn't say. The LLMs make it worse, but it's hardly surprising behaviour from them

> Their credibility was already building up ...

Don't you mean diminishing or disappearing instead of building up?

Building up sounds like the exact opposite of what I think you're meaning. ;)

Humans aren't very diligent in the long term. If an LLM does something correctly enough times in a row (or close enough), humans are likely to stop checking its work throughly enough.

This isn't exactly a new problem we do it with any bit of new software/hardware, not just LLMs. We check its work when it's new, and then tend to trust it over time as it proves itself.

But it seems to be hitting us worse with LLMs, as they are less consistent than previous software. And LLM hallucinations are partially dangerous, because they are often plausible enough to pass the sniff test. We just aren't used to handling something this unpredictable.

The kind of people to use LLM to write news article for them tend not to be the people who care about mundane things like reading sources or ensuring what they write has any resemblance to the truth.

The source would just be the article, which the Ars author used an LLM to avoid reading in the first place.

The problem is that the LLM's sources can be LLM generated. I was looking up some health question and tried clicking to see the source for one of the LLMs claim. The source was a blog post that contained an obvious hallucination or false elaboration.

If a human had enough time to check all the sources they wouldn't have been using an LLM to write for them.

[dead]

Another red flag is that the article used repetitive phrases in an AI-like way:

"...it illustrates exactly the kind of unsupervised output that makes open source maintainers wary."

followed later on by

"[It] illustrates exactly the kind of unsupervised behavior that makes open source maintainers wary of AI contributions in the first place."

Gen AI only produces hallucinations (confabulations).

The utility is that the infrenced output tends to be right much more often than wrong for mainstream knowledge.

You could read the original blog post...

[flagged]

The wrinkle in this case is the author blocked AI bots from their site (doesn't seem to be a mere robots.txt exclusion from what I can tell), so if any such bot were trying to do this it may have not been able to read the page to verify, so instead made up the quotes.

This is what the author actually speculated may have occurred with Ars. Clearly something was lacking in the editorial process though that such things weren't human verified either way.

Is there anything like a replacement? The three biggest tech sites that I traditionally love are ArsTechnica, AnandTech(rip), and Phoronix. One is dead man walking mode, the second is ded dead, and the last is still going strong.

I'm basically getting tech news from social media sites now and I don't like that.

Conde Nast are the same people wearing Wired magazine like a skin suit, publishing cringe content that would have brought mortal shame upon the old Wired.

While their audience (and the odd staff member) is overwhelming anti AI in the comments, the site itself overall editorially doesn't seem to be.

Outrageous, but more precisely malpractice and unethical to not double check the result.

Probably "one bad apple", soon to be fired, tarred and feathered...

There is no need to rush to judgment on the internet instant-gratification timescale. If consequences are coming for journalist or publication, they are inevitable.

We’ll know more in only a couple days — how about we wait that long before administering punishment?

I mean, I'm even more frustrated by this in Scott's original post:

> If you are the person who deployed this agent, please reach out. It’s important for us to understand this failure mode, and to that end we need to know what model this was running on and what was in the soul document. I’m not upset and you can contact me anonymously if you’d like.

I can see where he's coming from, and I suppose he's being the bigger man in the situation, but at some point one of these reckless moltbrain kiddies is going to have to pay. Libel and extortion should carry penalties no matter whether you do it directly, or via code that you wrote, or via code that you deployed without reading it.

The AI's hit piece on Scott was pretty minor, so if we want to wait around for a more serious injury that's fine, just as long as we're standing ready to prosecute when (not 'if') it happens.

I mean, he linked the archived article. You're one click away from the information if you really want to know.

I would assume that most who were journalists 10 years ago have now either gone independent or changed careers

The ones that remain are probably at some extreme on one or more attributes (e.g. overworked, underpaid) and are leaning on genAI out of desperation.

I work with the journalists at a local (state-wide) public media organization. It's night and day different from what is described at ars. These are people who are paid a third (or less) of what a sales engineer at meta makes. We have editorial review and ban LLMs for any editorial work except maybe alt-text if I can convince them to use it. They're over-worked, underpaid, and doing what very few people here (including me) have the dedication to do. But hey, if people didn't hate journalists they wouldn't be doing their job.