Comment by wxw
15 hours ago
> What you get is a beautiful animation that is 100% accurate and free of hallucinations.
I'm not sure I follow how this is actually guaranteed? The fact-checking process mentioned just seems to involve asking AI to review its own work.
All these LLM-as-review hype pieces don’t acknowledge that it’s turtles all the way down
What do you mean by this?
If the output can't be trusted, and you use another llm whose output can't be trusted to check the untrusted output of the first llm, then you're back where you started.
14 replies →
"Turtles all the way down" is a phrase of, I think, unknown origin (https://en.wikipedia.org/wiki/Turtles_all_the_way_down) about infinite regress or trying to patch up some bad theory by appealing to itself. Someone claims that what holds the Earth in place is that it sits atop a giant turtle, and a skeptic asks what holds the turtle up, and the response is that it's turtles all the way down.
Personally I think this is a bad characterization of using LLMs to fix up LLMs because while you can never guarantee results this way (as the quoted line claims here, which is worthy of criticism), it is, in practice, useful to use LLMs on top of LLMs. And there's no infinite regress. Auto-mode in Claude Code, for example, seems to me like it's been successful at making the system more safe than --dangerously-bypass-permissions without prompting the user for permissions constantly.
2 replies →
I read it as “if your LLM is being checked by another LLM, well then you need another LLM to check the checker. And can you really trust _that_ LLM? Probably should have an LLM to check the third one, and…”
It’s a reference to Bernard Shaw, who once said that if we ever created a truly artificial mind it would be inside a turtle’s shell. Sturgill Simpson covered the track on his seminal work, Xeno’s Paradox.
6 replies →
Other comments have answered in the concrete what was meant, but to answer in the abstract, in case you’re unfamiliar with the expression:
https://en.wikipedia.org/wiki/Turtles_all_the_way_down
[dead]
Agreed. Given how many significant errors LLMs make in my topic of expertise, despite my taking multiple error checking steps, the idea of catching 100% of hallucinations because you told the LLM to check itself is hilarious. It’s just a wild lack of insight: “I’m using the LLM to teach me something I don’t know about, I definitely have the knowledge base to spot any errors that might remain!”
Yeah. People with technical and/or tech business bonafides claiming that AI granted them expertise are so often taken at face value when they really shouldn’t be. Who told them that they were proficient — a chatbot? Someone who knows even less about the topic, so any expertise seems impressive? I’ll bet it wasn’t someone that actually knew what they were talking about. Even some tech reporters are tripping over themselves to be amazed, but don’t bother checking if they should be.
People don’t even have to be lying to be wrong about this stuff. Someone can learn enough about a topic to be halfway up Mt. Stupid in no time flat, and in doing so, think they not only truly understand the topic at hand, but might be particularly adept because they were such quick studies. People that know less are impressed, because why wouldn’t they be? Anybody that knows more than them sounds like an expert. And people that know what they’re talking about cringe at the overconfidence, and probably try not to engage: who wants to have to prove that someone’s boundless confidence is entirely baseless? Most of the time, they think the actual expert is full of shit because they think they’re the expert. It’s incredible how many times I’ve had people in tech confidently, even smugly “explain” design concepts and strategies to me that they did not actually understand, knowing I was an experienced, degree-holding designer… and they didn’t even have a chatbot’s lips on their ass telling them how smart and insightful they were.
[dead]
Completely agree. While I didn't set things up to have AI review its output in a loop, my experience trying to build a specific acoustic testing rig with Opus 5 also aligns with the other "it's turtles all the way down" comment.
Opus 5 first built me a detailed plan, but a couple important details were either obviously wrong or felt unnecessary. I went back and forth asking for sources and more information probably like 4 times and every time it did the "in looking at things in more detail it appears my previous advice was incorrect" spiel. It just became exhausting at some point because it feels like it really lays bare how LLMs are just minimizing that loss function but don't actually "understand" anything. It was really useful as a search engine (it correlated some highly relevant source docs), but I just couldn't trust it to believe it was actually done at any step.
A second agent reviewing it adversarially resolves some context rot. Whatever they trained these LLMs on will just infinitely double down so I break it with 1 layer of checking and then a judge who looks at facts, since the checker is adversarial.
I know it sounds silly but 1 layer ends uo being way worse than 2.
I could certainly envision a scenario whereby review would increase reliability but not how it would every guarantee 100%, there is a pretty big logical gap there.
In my experience it depends on how much in detail you want to go. Chip manufacturing is a really opaque industry, so in this particular case LLMs might not even have the training data. However, using it for a high-level introduction into something is usually pretty safe from hallucinations.
even if you say use RAG or something to a source you can trust, there's no guarantee the agent will still use exactly what the source has.
i can't even get agents to remember core instructions like "use jq instead of writing a python script to parse some json"..
The "100% accurate and free of hallucinations" claim should probably be replaced with something much weaker
I don't do animations, but I have an answer. You research a topic well enough to be able to understand if the result is OK or not. Usually it means figuring out some sort of testing.
I'm researching causal inference right now, and my main goal was to make sure I understand how to test estimation on synthetic data.
Basically, it's the same way it works with people. If you delegate a task that you don't understand, and you can't have a credibility proof (i.e. doctors, lawyers), then you research a topic well enough to be able to (1) define the task and (2) verify the end result.
You can add "make no mistakes" to the end of the prompt and achieve the same result while burning less tokens.
Yep, that's impossible. The hard truth is that most people this lost to LLM psychosis cannot understand that fact. It's better to treat it like someone in a cult, arguing the facts isn't going to help if they refuse to accept them.
I thought after reading the title that the text was about learning something, yet the actual text seems to be about having a system do something for me.