Comment by Hansenq
7 hours ago
61% AI generated, according to Pangram https://www.pangram.com/history/e5a00ace-94cc-436e-b87b-a094...
c'mon, you're Carlyle, a trusted institution for financial advice! How can I trust what you're saying if the AI-generated text is so blatantly obvious?
Is there any reason to trust Pangram's tool over Carlyle's reputation? I don't know whether this was AI generated or not, but an online tool claiming it's AI generated doesn't sway my opinion much. Are there any studies showing that Pangram's tool is well calibrated for this type of article? If not, what makes you trust it?
(I'm not sure why it was marked that way, but I vouched to bring your comment back from auto-dead. I glanced at your comment history, but don't see any clear reason for this.)
I only went to Pangram because I read through the essay and could not help but notice the Claude-like sentence structure and AI-isms in it, which distracted me and completely made me distrust the thesis of the piece.
My understanding is that Pangram is the best out of all of the AI detectors and if there is a better one I'm happy to switch to it. And it's easier to point to them than to give explicit sentences and examples about why it reads so AI-generated (and since you want it, it's these sentences in particular: "The message was unambiguous: energy is finite, security is earned, and comfort has a cost.", "That template is being applied again today — and markets have accepted it.", "One country never made the West’s mistake.", etc.)
Reading through it again, there are so many emdashes. And don't get me wrong, I was a liberal user of emdashes before AI! But just like, if your job is to communicate your thoughts to a wide audience at least respect their intelligence enough and not rely on crutches just to get an article out against a deadline.
To me these look like professional writing, and I've seen worse LLM'isms. I am with some people in that I don't usually want to read pieces where GenAI was involved. But this is one of the cases where I wouldn't want the remedy to be a life of constant purity tests and suspicion. There has been enough of that in contemporary culture. I want to trust people and let them be by default. I mean, I don't want writers to specifically memorize current LLM'isms in order to avoid them: or worse, to prompt an LLM to strip them. I believe from experience than GenAI texts have secondary bad characteristics, mainly being bland in style and thought, and often going nowhere, like mandatory student papers. This should be enough to bury or criticize them on merits more often than not.
I do see myself treating slightly broken grammar, typos as a slightly positive signal in writing in the recent years. I also see that this is messed up - I'd like to respect someone in kind if they put care in writing - and easy to fake if anybody wants to, with an LLM. If you do strongly suspect GenAI writing, I mean it's fair if a sincere opinion, but I'm tired of having those as top comments with a whole response tree. Ironically contributing to that now.
It’s either heavily AI written or the authors style has been heavily influenced by LLM writing. That said, using LLMs when writing does not mean the article is not informative, thought provoking, or relevant. An idea fleshed out in prose by an AI is not the same as AI generated content. But it is lazy writing.
How do we create an environment that incentivizes human writers to focus on their craft instead of “productivity”?
Thats maybe one of the more pressing issues of our time.
Yes there are studies, for example last year Pangram's false positives were measured to be under 0.5%.
https://www.pangram.com/blog/third-party-pangram-evals
Personally, at first I thought these sorts of tools were dumb and wouldn't really work, but I think it works because it just isn't designed to be "adversarial". If you want your AI to trick Pangram, you can make an AI to trick Pangram. It just catches people who are cutting and pasting from the AIs without putting any more effort into hiding it.
Any binary classifier can have a FPR under 0.5% if you don't have any restriction on FNR...
2 replies →
> Is there any reason to trust Pangram's tool over Carlyle's reputation?
You have your own chatbot, right? Ask it. I've never had one disagree with GPTZero yet.
Is there a particular part you are claiming is incorrect? If so, point it out and name your sources. If not, what’s your point?
I do not understand why this matters. Judge the content on its merit. It makes no difference if “AI wrote it”.
This has the same energy as people who don't care if controversial images in social media are AI generated, as long as they're engaging.
It makes a huge difference if the writing was manual or automated. LLMs generate verbose, generic writing, and ideas that could be concisely expressed in a sentence inflate to entire paragraphs. It's disrespectful to readers when the author saves a couple of hours by wasting thousands of their readers.
Article from a trusted organization is supposed to be grounded in real-life events and filtered through their specific area of expertise. They bet their reputation on it, at least in theory. Current agents have none of that, and you can't trust their output to the same extent in practice. Too much recognizable slop in the article suggests high degree of autonomy of the agent that has been used, and raises doubts about trusting the entire article. It might be the valid opinion of the author rephrased by the model, but you can't tell as it obscures actual intent and the amount of real life data.
If AI wrote it, it is not worth effort engaging with.
Nonsense. Of course the tool matters.
That's why I only use code written in Vim. Emacs corrupts the othewise identical bits, just like AI. Gets 'em all greasy and then they smell funny.
Get me some holy water
I wasted some time trying to get a sense of the content, to "judge it on its merit". It is a total slop. And if i knew beforehand that it is AI, i'd have spent much less time as the first short look would have confirmed that it is slop.