Comment by davidgh
8 hours ago
My son is currently learning Romanian and I was trying to help him with verbs. I don’t know Romanian but recalled when learning a foreign language for the first time it really helped me to break down how a verb form or tense worked in English, then learn the equivalent in the new language. So I wanted to make some charts and pages that he could use as learning resources.
I used Claude to help. I don’t know how to quite describe it, but because the text was polished and well constructed my brain was giving me the the signal “if you aren’t getting this it’s because you’re not focusing” so I’d read it again and then again and it still was not landing. It sorta felt like when you read something technical or heavy when very tired - you are reading but not processing.
Only after wrestling with this for a few days did I realize that it wasn’t me. As I started going through, sentence by sentence, forcing it to re-write things to be more clear the concepts became easy to understand.
I wish there was a name for this situation. It’s almost like a pseudo-language where it has the correct form and presentation but is missing critical components.
The more complex the topic, the more I sense this.
This is a really good description of the problem. I’ve been trying to use Claude to get familiar with the mechanics of a new codebase, and there have been so many moments where I’ve stopped after reading the same paragraph five times in a row and thought “Am I tired? Or stupid? Or is this codebase just wildly more complex than anything I’ve seen before?” before realizing that it’s just taken English and smushed it around like a ball of clay into some abstract sculpture that kind of evokes something from real life.
I think part of it might be an innate feature of LLMs, but Claude seems extra prone to it lately. I ran the same query about the same codebase with Codex, and it gave me an answer that was about 1/4 the length and made me realize that it really wasn’t all that complex.
If nothing else, it’s good training for my own writing. I’ve been working on making myself be more straightforward and concise, and Claude’s writing is a good example of how cleaner prose is a functional choice, not just a stylistic one.
> but is missing critical components.
The human spirit. When you read a real person's thoughts you can often intuit the thought processes that led them to write it which aids understanding. Or at least have a general idea of "where they're coming from". But an AI is missing that. It just knows everything, without a "thought process". Instead of a flawed 3d person, we get a nice 2d picture instead.
Feels like the way a video game will render the outside of a wall or solid surface, but you can run into it and warp partly through and there's nothing internal to it at all.
> I wish there was a name for this situation. It’s almost like a pseudo-language where it has the correct form and presentation but is missing critical components.
The best I’ve heard of this is peeling the onion. The first pass is always very high-level and you have to make it go deeper. That can be done manually with follow-on prompts but I like using subagents, each with a different angle on the problem.
> It’s almost like a pseudo-language where it has the correct form and presentation but is missing critical components.
Slop. The word is slop. Has been for years now. I mean, is this not exactly what we've all been talking about the whole time?
> I wish there was a name for this situation. It’s almost like a pseudo-language where it has the correct form and presentation but is missing critical components.
Bullshit?
I understand the sentiment, but not quite what I’m thinking. It’s not that the AI is necessarily wrong, it’s more like, it replaces clarity with rich but unhelpful text. Maybe like candy. Full of flavor, texture and color but lacking the nutrients.
The way I like to describe it is that it feels slippery, like it's been polished down to oblivion and my eyes slide right off of it. There's no place to find a mental foothold, and if you zoom in there aren't any details.
But I also like your candy analogy because I think it's spot-on for how LLM text superficially looks informational/nutritious, even though it's actually just junk.
No no no. Harry Frankfurt wrote at length the difference between liars (who care about the truth, and twist it) and bullshitters (who don’t care about the truth, but just the way they’re received).
This is some third category of untruth. Almost more sinister than the other two altogether.
"A third category of untruth" is a great expression, thank you.
A lot of it is hedging and using lots words to avoid saying something that isn't true. That way, it looks a lot like text that contains meaningful information, even though it doesn't. The speaker can pass the scrutiny of an informed audience, because they are able to substitute their knowledge into the words, as if by pareidolia. It's the sort of 'diplomatic' language you see from politicians, lawyers, students writing essays, and any other moderately intelligent person who is put in a position where they will face consequences for not answering. Behind all the fluff, there is a very loud voice yelling "I DON'T KNOW."
I think, they don’t care about truth at all, just whether the reviewer rates it highly. Otherwise, it would be deleted after that round of training. It values test-taking ability over critical thinking.
My two favourite words for this are “conditioned” and “catechized” where the latter is a bit more on the nose but way more obscure.