Comment by marcindulak

2 days ago

Here is an explicit example of the additional lack of coherence due to LLMs in the Anthropic Claude Code public repo itself: the human maintainer trusts LLM's incorrect understanding of the current state of code in the repo.

The comment https://github.com/anthropics/claude-code/issues/30407#issue..., written by Claude Code, and posted by the human GitHub account says:

> "Issues here are auto-closed by a bot when they've had no activity for a while and are missing information needed to reproduce (usually a version and steps, or a /feedback ID). Nothing is triaged solely by counting"

In reality all issues in this repo are subject to time-based auto-closing, unless they collect a number of upvotes. See details at https://github.com/anthropics/claude-code/issues/87647.

Therefore, I wonder whether the internal review process described in the Anthropic's blog for internal repos also results in such issues, and whether Anthropic has a internal way to measure and mitigate that. The blog post says:

> "When someone wants to understand a part of the codebase, they ask Claude to write an explainer for exactly that slice. It takes minutes and reflects the code as it is today."

The Claude Code public repo problem shows that LLMs don't always "reflect the code as it is today", and can be a source of additional misunderstanding.

Exactly this. Using an LLM to understand the original LLM slop is just another chance to get it wrong 20% of the time. These things just aren't reliable.