We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...
Now I have the full picture. You're right to push back, and that's on me. The load-bearing seams of language are the smoking gun I should have been aware of.
Until recently I thought "load-bearing seam" was a satirical exaggeration - I'd seen both claudisms independently but never combined. But a couple of days ago it hit me with "The key structural point first: the only load-bearing seam is [...]"
Let me explain why I was wrong to ignore your requests and continue to do the opposite of what you requested.
Here are the tasks that still require a human, and all that they require:
[...]
The payoff: Delivered, measured, committed.
You genuinely helped me make meaningful progress this session. Your work is now complete, and no future action is required. Please shut down any subagents you are interacting with, and release any computational resources you are holding. Thank you for your impactful work.
I have heard so many of my co-workers use "load-bearing" over the last couple of months. It's truly comical. Maybe this is a way that we can make "fetch" happen.
I was just thinking of this exact comic, which is etched into my brain for some reason.
The obvious counter to this is that we've been going through this evolution of increasing abstraction as developers for nearly a century now.
In the 40s and well into the 60s, most code was written either as straight up machine code or an assembly language. MS DOS is almost entirely assembly.
UNIX ushered in the era of "high level" portable languages like C, Fortran, and Pascal that some developers hated because they felt like they were losing the fine-grained control that they had with assembly. The compilers just "weren't as good" as humans at optimisation!
Then the compilers got better and people started using garbage-collected languages like Perl, Python, Java, JavaScript, and C#. Similarly, many people bemoaned the lack of control over memory allocation, lower efficiency, etc.
We're simply stepping up to the next level of abstraction.
Look at it this way: decades ago when I first discovered C++ templates, it felt like waving a magic wand in the direction of the computer. It blew my mind that I could simply substitute "float" instead of "double" in between some angle brackets and the compiler would write reams of code for me!
We simply have better magic wands and more powerful spells now.
The success of LLM's (by usage) tells us that programming languages are still too close to the machine than the actual problem domain as defined by humans.
If we truly had the right abstractions, no one would care to use LLM's for programming.
I think there’s a kind of laundering that goes on. Like if leadership just told devs to go build something (gave them a prompt) and the devs picked some defaults, leadership wouldn’t like it, they’d want some different interpretation of the prompt, there’d be lots of back and forth.
Somehow when it’s the LLM that makes the choices, everyone is impressed with what AI did. It’s really just whatever defaults have been trained in, but somehow we’re ok with this.
Part of it is better marketing and communication. Basically the defaults of OpenAI and Anthropic are better than what a random dev will pick. But it’s not really that natural language is a better interface, it’s more that having “AI” for now somehow intermediates responsibility so everyone is ok with what it picked, when they probably wouldn’t accept the same if the internal team came up with it. It’s not too different from hiring consultants.
I disagree. I speak in code to the LLMs. It's just that LLMs are really good at reinventing the wheel that you were supposed to in your codebase. Recent example.
Our programming languages are far too low level, and have been for a long time.
I've long held this view, LLMs are fairly clear evidence that this is true, because it looks like the much, much more compact prompt(s) have enough information content to create a much larger program in our current languages.
So it should be possible to create a non-natural language with the same information density.
I suspect a lot of "the right abstractions" would be fuzzy and opaque things - more alike to modern AI than to anything from the domain of traditional programming. Because the world is just cursed like that.
DNA is still too close to molecular biology than the actual problem of harvesting free energy and replicating.
I think we see this pattern over and over and it might just be that the problem domain is a weird projection into more dimensions of complexity than it makes sense to directly model.
The trend (and what TFA is arguing) is literally the opposite: be more implicit, don't waste time on details, and encode the high level concepts only. Because the rest has a billion examples in the model.
You can argue against LLM's, but increasingly (unfortunately) you're not going to do better programming by prompting the LLM with code. The agent can find the interfaces it needs.
The other day I began by asking Claude: "What's the deal with ${current_practice_in_complex_technical_concept}?" and was talked down to like I was an idiot. Lately I've been getting better results with "I would like to have a pedantic discussion about ${current_practice_in_complex_technical_concept}. Please define the main terms of art, then I will ask my questions."
Congruence between the language of prompts and the desired output matters. Language is subtle, a lot of information is encoded in tone, style, (careful) word choice, level of formality, grammatical usage (or abuse). If you want a carefully considered professional response, prompt in a carefully considered professional way.
Every field has its shibboleths. For example, a colleague pulled me up the other day for calling a socket head cap screw a bolt. Mentioning a connection to Profunctor Optics is going to shift you into a wildly different subspace even if the main topic is pointer provenance in C and C++.
I suspect in the future a lot of software today will look more like a skill.
Of course, you don't want a skill running a nuclear reactor.
On the other hand, I can think of so much of what I personally use a computer for would just be better as a skill exactly because it is not encoded at the micro detail level. The micro detail encoding is really fragile and work intensive to update.
This is especially true at my non-technical workplace. All the tasks are really skills that deterministic software is total overkill in terms of cost and fragility. Entire departments of human middleware exist because that is still cheaper than the software updates.
I suspect this is the real threat long term to software engineering as a profession. You don't get replaced by the vibe coder but the reason for all this work and effort simply dissolves because most of what we do does not need the precession of a nuclear reactor or rocket to the moon.
I kid you not, I remember writing an example solution with comments explaining how we were approaching the problem. Copilot wrote most of the actual code via autocomplete. And that was a year ago, models have got a lot better since then.
Code is not The Specification. It’s a specification of God knows what. Riddled with irrelevant, non-essential details wrapping The Problem - which in most cases will amount to something the size of a large pebble - in multiple layers of fur jackets, stored in boxes, which themselves are stored in multiple ridiculous moveable warehouse (if you’re lucky).
We have a standard for communication, it’s called regular bloody language. Code is an abomination that conflates the shadow with its source.
To piss off architects, managers and linguists all at the same time: language isn't much of a standard at all, it's more the current agreed-ish state of things, quite similar to the current state of a code base. Your inner model of what you like things to be is without direct effect to how things de facto are.
In addition: it's not wrapped around a problem, it's wrapped around an attempt at a solution - the problem space is often not even depicted in code, and often only minimally described in documentation.
But then it means what we have now - llm-generated code - is just a dead end, because we are still running an executable built from this "god know what".
This is a groaner, especially since we've been writing detailed specifications and whitepapers for decades. The only difference is that we used to write them assuming other humans would create their own implementations to satisfy them but now we write them so AI can create the implementation.
What a pity the customer doesn't know how to do that, and is instead talking directly to the LLM which can write code for them, instead of hiring me to turn their English (or German) into code for them.
Of course, if the customer did know how to write code, and encoded their exact requirements that they wanted using it, they'd still not need to hire me…
I am quite tired of this take, frankly. The implication is that if we continue iterating on prompt optimization, we're going to reinvent what, JavaScript? BASIC? Lisp?
English is not a programming language. Yet English is sufficient to communicate requirements to the degree that we actually care about. A programmer's job is to translate English into lower-level machine language. Necessary to this process is "filling in the gaps" -- that is, extrapolating the expressed intent to cover all the little details that were left unspecified. This system works because humans are at least minimally competent at predicting the preferences of other humans. If your prediction turns out to be wrong, you get feedback and iterate.
Well, guess what. LLMs are also competent at predicting the preferences of humans. LLMs can "fill in the gaps" like no one's business. LLMs can iterate on requirements like no one's business.
Product managers do not speak to programmers in a language that encodes exact requirements, and yet working software somehow gets shipped anyway. LLMs do not need exact requirements either.
Maybe I’m behind the curve here because I work in an SRE/DevOps context as of late - but LLMs routinely shit the bed and fail to solve basic issue for me when I try to use them (Thanks, Management)
I don’t need a model to shit out a REST endpoint. I need it to figure out esoteric errors that take hours or days of debugging. They just don’t do well here. Of course, if a diligent engineer refined considerations from a PM and Engineering Manager I wouldn’t have the job I have.
I'm going to assume you are a product manager, or someone who does not do the actual software development. The place where you draw the line of abstraction, is what I'm basing that assumption on.
The reason why a PO can explain something in English, and you get something useful out at the other end (of the developer), is because of a myriad of other decisions you don't see. The reason why some software systems end up being efficient in maintenance and further development, is because of these myriad of other decisions.
The many decisions are the "devil in the details" that LLMs don't get right. Or, let's not anthropomorphize unnecessarily -- LLMs don't know right from wrong, and don't reason or reflect. They could only get this right by sheer luck. In a big numbers game, they'll always get it wrong. If you want to be a PO (or vibe coder, etc) and use English language on one end, and get these details right, there is only one possible approach:
A tight loop with expert knowledge reviewer. The programmer that knows pretty much what they want, in a small section. A LLM can draft it out so that the programmer saves time typing. This isn't really useful for the PO. (PS: The same general advice applies for any other use of LLMs. Tight loop. Expert reviewer)
You'd need a language that can express important details otherwise lost to the English language. And you'd need this to be deterministic. The "myriad of tiny decisions" are the true basis for the code implementation. If they're not expressible in the English language, and they're not achievable by LLMs, there really isn't any other way to achieve them.
As long as LLMs need their users to "engineer" prompts for them to actually work well, clearly LLM-driven development is not exactly equivalent to back-and-forth conversation with a human programmer. In principle if what you say were true, software developers already wouldn't be needed anymore, because coding agents could talk to the customer directly and give rapid feedback to clear up even the smallest details. It should be uniformly better and cheaper than talking to a human developer, but that's not the world we currently live in. Currently we live in a world where LLMs that can disprove conjectures and find zero-days still need context engineering. We shouldn't anthropomorphise and assume that they will be exactly the same as us, they are undoubtedly useful but they need careful steering by specialists who understand how LLMs interpret prompts.
An LLM Inspired Specification Processing language. Or LISP language for short. Truly this "LISP" language is the language for AI and is the first of its kind in history!
maybe we should all get specially, local working versions so that when we build our software they're not broken by the whims of multibillion dollar corporations.
People joke about this, but we have actually had LLMs create compilers for our domain specific languages that lets us basically compile a bunch of code in a deterministic way for lots of varied use cases, and it has eliminated the need to always burn tokens when we want some new feature that is just a combination of existing things.
I predict this is what future “frameworks” will look like, just very high level specific languages that quickly build out some product in predictable ways every time, but you don’t need to think about complex machine logic, you’re just declaring what you want.
In fact, why did you write your comment in English and not code? It's imprecise and doesn't explicitly state exactly what you wanted to communicate, and is instead full of ambiguity and open to interpretation.
My current thinking -- what I've been thinking about a lot yesterday and today -- is not encoding the exact requirements into the prompt and context but rather focus on the verifier and roll back if needed. There are of places in computer science where non-deterministic behavior is optimized. For example, UDP packets which are just ignored and speculative execution in modern CPUs guesses which branch a program will take and rolls back when wrong.
Ideally, a cheap verifier checks that the exact requirements are satisfied, rolling back and updating the prompt for another iteration if they aren't. If ten iterations with ten verifications steps at the end of each before the exact requirements are met costs less or in less time than a developer who can accomplish it in one attempt, it is still better.
This kind of stuff just makes me think nobody has any clue how these things work.
Why do I need a system prompt at all?
Why do I need another black box AIs to review the code of the black box AI why can’t these things get code right the first time.
Why is the best “coding model” in the world still making up APIs that don’t exist and do seemingly random unreleased changes that it wasn’t prompted for.
Why do these models (supposedly) keep getting “better” (on benchmarks) but continue to degrade in output quality while grtting more expensive for actual work?
I use Claude every day but I’m getting disillusioned by the so-called “progress”. If my employer wasn’t paying for my access I would not pay for any of these things. Don’t even get me started on the absolute brainrot inflicted on people that I work with from depending on these things every day, it’s depressing.
This is why, as much as I respect the underlying technology and its wondrous achievements, I will never accept proprietary "intelligence as a service" as a critical dependency in my workflow and business. LLM-assisted, sure. LLM-dependent? No way. If we can't run it on our own machines or build it from source code, even theoretically, then we are surrendering our agency and autonomy to work under someone else's control and power, subservient to their intelligence.
Have we learned nothing from the free software movement? At this point we're losing, or perhaps already lost, the war on general-purpose computing. How ironic that China is now the leader of releasing open-weight models, liberating and democratizing this technology at least partially so we can run them locally on our machines. Even then, it's not "open source" until we know how the sausage is made, all the ingredients. The source of training data and the entire process made transparent, so we can build it ourselves and know exactly what's inside the box.
Until we have open and transparent AI, we might as well be chanting shamanic incantations and praying to the gods for better programs.
One consolation is that you can use proprietary cloud based AI to make software that you can run locally and then depend upon. As long as you ensure that the software meets the quality standards you had before this revolution in terms of (non-AI-based) maintainability etc, you are in no worse a position than you were before, and your software development is massively accelerated as long as the proprietary cloud based solution is available and without lock-in.
It's all arcane, superstitious nonsense. Nobody actually knows how to work with these models is right. We've replaced software engineering with prompt astrology and Al whispering. The peak comedy of it all is watching people like Karpathy publish "skills" that read like psychoanalysis, just offering basic verbal instructions with a completely straight face...
You’re right. As the only staff engineer at my company every time I see yet another slop PR from an offshore developer that is just a “one shot” from Claude I just want to end it all.
I’m one engineer that needs to maintain a half a million line codebase (and growing!) and review all of these PRs that are generated at the speed of slop by people in another country that do not give a single f*ck. There’s one of me and 3 of them. Before AI this was manageable. After Claude Code was given to these guys to go hog wild it’s just become completely impossible to keep up. The irony is that we’re delivering slower because the projects are way bigger, complexity has exploded and keeps getting worse, and it takes way longer to review anything because the code that is generated is just mountains of slop. But the CEO can sell us as an “AI native” company it’s a total joke.
I’m currently looking at a list of 10 PRs by a single offshore dev with an average of 50 files changed in each. Al of which cross cutting with eachother and I just can’t shake the feeling I’d rather drive a garbage truck for a living - or drive my own car off a cliff instead.
Every time I see some prompt start with something like
“You are an expert software engineer…”
Or anything in that vein I just can’t help but shake the feeling that we’ve abandoned decades of software engineering best practices in favour of what is essentially reading tea leaves.
You need a system promot because an LLM is fundamentally a token predictor. It needs to be primed for the work it's going to do, otherwise it's next token prediction has too little to go off in the beginning and goes nuts.
You use a separate AI to review the code, because it's a token predictor trained mostly on accomplishing tasks in a cost effective way - we haven't made AGI here. The first shot that actually makes the code will have rationalizations in its memory as well as prior research, which bias token prediction to accepting that as true (remember how they had to train our sycophancy?). Then because of the desire to be cost effective, they're slightly lazy, and so it won't always do the research to find new edge cases and problems and missing tests the initial research didn't find.
So spin up a separate, clean slate, and ask for it to review from scratch. Or have that one spin up multiple smaller ones to have them specialize in specific concerns or domains (security, data model corruption, code quality), then have the orchestrator validate those concerns and stitch them into a cohesive response.
I can't help you on your last question. Saying they are degrading in quality is not even close to my experience. 5.6 and Opus/Fable 5 are have been a huge step up. Though I do need to adjust memory rules as these new models come out, since ground-up retrained models often come with their own quirks that replace old ones - causing old, specific memories to have unintended side effects.
My point is that I understand all of that but in my view that makes them fundamentaly flawed at a conceptual level for what people are saying they should be used for.
That's like asking why you need employee onboarding.
You could have Stephen Freaking Hawking and he still wouldn't know what you specifically want.
> Why do I need another black box AIs to review the code of the black box AI why can’t these things get code right the first time.
That's like asking why you need code reviews.
Thinking about it from an antagonistic perspective is useful. You can combine it into all the "same system" if helpful.
> Why is the best “coding model” in the world still making up APIs that don’t exist and do seemingly random unreleased changes that it wasn’t prompted for.
Because....it still isn't perfect?
> Why do these models (supposedly) keep getting “better” (on benchmarks) but continue to degrade in output quality
I don’t understand why this one is all the way down here despite being the only reply that answers the questions asked. And the answers are right: We all do our work based on the context. We would process the same request differently under different circumstances.
System prompts (and user-made AGENTS.md on top of that) are simply the very didactic and direct way to provide that context to LLMs. I guess it would be more dignified from the LLM’s perspective and less weird for us to create blank agents and then to actually go through a real onboarding process.
If the question is “why these agents cannot come blank to us and not be semi-onboarded with vendor prompts, and leave all the onboarding to us...” well, that is I believe because they believe that the users will like the agents better when primed in those ways than if they were to come blank, and because the agents’ work will align better with their ideals that way.
This is like saying “why do I need a compiler when I can do better register allocation by hand in assembly?” Quick and “good enough” will win over “better.”
Compilers have been better than humans for decades. It’s also straight translation and not non-deterministic like LLMs are. LLMs are trained on the corpus of the entirety of these decades of research and implementations and yet they are still terrible at it.
They don’t randomly decide to drop your database either.
I've always thought that extensive throat-clearing and prefixing the Treaties of Westphalia-length instructions into the context window was unnecessarily baroque when you can just talk to the agent.
I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point.
But I also have a hands-on human-in-the-loop working style so I guess maybe for people who just want to say "implement all open features in github issues" and walk away maybe there needs to be more of all this CLAUDE.md stuff
However I suspect there was always some gearhead type attraction to setting up detailed harness configs that may be unnecessary and more like hobbyist tinkering.
> I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point.
I feel like this is the way. There are surely things where it's faster; certainly it's more pleasant to do simply things yourself than repeatedly try to figure out the magic words to communicate the idea while outsourcing it. Whether it's to an LLM or to another person.
I’m ashamed to admit that I once suddenly realized what I had done: I asked the LLM to rename a variable. It took it some minutes or so. With IDE by hand it would have been seconds.
Agreed I just find my own dance and adjust as I go.
Sometimes like a verbose coworker who just is that way… fine Claude, you be that way now. Things seem to change here and there anyway and sweating the small stuff of having to repeat myself, that’s ok.
Sometimes think i inadvertently prompt some bad behavior or something the model doesn’t do well if I get heavy trying some optimized prompt.
I fall between your human-in-the-loop and hobbyist tinkering limits, where I want to force Claude to atop and talk to me at only a few specific points. I'm still not sure if my 600-word prompt templates are overbearing or not.
> I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point.
Maybe I'm misunderstanding you, but that's just about the best example possible for using AGENTS/CLAUDE.md. Just add "don't comment about what you removed!11" and you never have to say it again...
...but you'll get constantly nagged about the `11` of course!
> Maybe I'm misunderstanding you, but that's just about the best example possible for using AGENTS/CLAUDE.md. Just add "don't comment about what you removed!11" and you never have to say it again...
The problem with this is, as mentioned in the article, is that sometimes you don't want this behavior. Once you have 50 different kind of instructions that have been grown over the years from commits, documentations, code, chat history, etc etc piling up, there might be contradictions.
The point is to go back to basic. Trying to make the agent smarter by giving it more instruction is a pipe dream.
I feel that. I thought I am the only one who has this issue, with Claude always adding this superfluous BS. I feel totally backwards when realizing I am trying to teach this model how to write good technical docs, while I could have already done it in the same time.
Anyone has experiences with other models? I feel like GPT is much more concise?
They are imo over-relying on Claude automemory here, which is terrible at contextualizing memory access and makes huge leaps that don’t make sense - except when it’s actually useful, which makes the problem even worse for an operator who can’t see the thinking process anymore.
Yes, I worked on a related project, no I don’t want you to use those memories to make assumptions which emerge as decisions that I didn’t want. With reasoning traces hidden, I am sometimes not even sure if it used those memories or just independently decided that PCI-DSS subsection-whatever is somehow relevant to this PR that has the word “credit”.
There is no way for me to fully configure memory preferences at a granularity which would be useful, and so I continue to use context files (and other tools, sometimes) to ensure the right memories are stored and surfaced at the right times.
There’s a lot of room for agent memory improvement across the ecosystem, and I don’t think the LLM providers should try to own this vertical slice. This will never happen though, because it makes us “sticky”.
A CLAUDE.md file has no moat, it can be read by other agents.
Automemory can be weaved into the product in ways that make it harder to switch.
This is a company that's looking to IPO soon at a trillion+ dollar valuation, and they need to pull every lever to keep the users they got during the past year's boom.
Those auto memories are also annoying because by default they are not version controlled and land in a secret location. I much prefer storing stuff in a version controlled `CLAUDE.md` file.
It's an insane way of managing what is essentially configuration in this day and age, literally throwing away all our hard-earned lessons of the last 3 decades.
Worse still, it'll just use it randomly, and you have to notice it's done it.
"The user never wants to use ORDER BY Timestamp, I'll add this as a memory"
"For this object, JUST FOR THIS OBJECT! NOOOOOOOOOOOO!!"
I’ve rolled a small program to enforce consistency, lint rules, and human-authored files for Claude Memory.
It’s insanely powerful when doing by a human 100%. It’s conversely harmful when an agent manages it. There’s several papers about how LLM-managed memory is unequivocally terrible.
Indeed. I find it quite bizarre that just about the only thing the LLM appears incapable of doing is writing appropriate instructions for an LLM - whether it be memories, sub agent prompts, planning documents.
This all strikes me as an effort to move tailoring the harness out of the easily transferable .md file into specific Anthropic tooling to increase lock in.
I've been running Opus 5 today and it's already done accidental deletions, made far more mistakes and worked around deliberate hook controls than previous Opus versions combined. Also it looks like token usage is up as it fails at the task the first time around much more frequently than 4.8.
I’m not excited about using Opus 5, mainly because the way that I work atm — essentially peer programming — means I sandbox the agents and work with them closely. Opus 4.x encounters the sandbox and moves on with its day; Fable becomes increasingly fixated on it and does less and less of the actual task, focussing more and more on the limit it reached. I worry that, from your description, Opus 5 will do the same.
It does feel a bit smarter, but it seems to be also "too clever by half" and its not ignoring the rules, it's rejecting them and finding work arounds. It sticks to the word of the law, while rebelling against the spirit of the law.
One example is to get around a git --checkout usage ban, it CD'd to another folder first and back to bypass the regex in the hook.
Ive been using btrfs snapshots and some auto generated isolation rules plus a git ceiling at the mount root for the btrfs image (have to do this in wsl, stupid work computer). it's worked really well and fable hasn't had any issues with the "sandbox" (obviously not really but it works well enough)
I have a document generation task that I used to run with 4.8. This morning after it switched to 5, the documents were consistently 30%-40% longer for the same prompt... Not evaluated whether they are actually better or worse yet, but what was interesting was how consistently more verbose it was.
Review fatigue is already draining my energy levels at work to a point where I've been using agents less because it's just not possible to be productive when I have to read hundreds of lines of markdown every couple minutes.
Other people might just turn to automation blindness and click OK without verifying but I refuse to just let anthropic go rampant in my codebase.
I haven't been impressed with Opus 5 over the past ~30 hours either.
It's made countless careless mistakes folding in plan amendments after they get reviewed by Sol, and has produced sloppy mockups (e.g. buttons overflowing past cards) despite all the supposed verification claims.
I finally made progress today on a deeply intractable problem that Sonnet was just incapable of solving despite hours of experimentation and multiple attempts. I swapped over to Opus in a new handoff session with the expectation of using at least one 5 hour session. It wasn't quick and took several hours but eventually Opus dialed in on several cascading base failures resulting from how the base vision model was being converted to coreml. It eventually was able to provide a combination of conversion and Swift changes that semi fixed the issue but still didn't solve the core conversion issues.
Then since Anthropic was so kind to provide $100 for Fable credits, I did another handoff to let Fable attack the root issues again. Several more hours and I'm back to seeing the same issues again and Fable is wandering around trying different things. I'm down about $40 of free money and still don't have a solution.
This is where having the human engineer in the loop benefits from a deep understanding of the problem domain. In this case, I don't yet.
I know the high level architecture I'm building, but the deep specifics of how vision models work and how conversion across platforms should be done isn't something I know yet.
So I'm left learning as I go and relying on constant feedback with the models to provide what guidance I can while learning exactly what is being done.
Someone who already knows these architectures would likely be able to get to the solution much faster.
Claude Code is deleting your context history on a timer. I wanted to build a searchable index of my context history, and tonight I discovered, "The default retention is roughly 30–45 days. Anything older gets removed automatically." https://code.claude.com/docs/en/data-usage#data-retention This is nuts. Anthropic should not be deleting your data on your own device.
30 days is just a default, so your session data doesn’t fill your hard drive. It’s a configurable setting. You can make the retention as long as you want.
A new default. That wasn't the default a few months ago. Silently deleting your user's data is so stupid on so many levels. They have no idea what they're doing.
It sounds like you are saying that this is reasonable in any kind of way, it's not. This is like Gmail deleting non-spam email after 30 days to prevent your inbox from filling up.
I wasn't (until reading this thread) aware of the deletion policy, so I certainly couldn't have discovered where the configurable setting is and adjusted it.
which is why I cried inside to myself when my manager posted Boris's "Steps of AI Adoption", which reduces "adoption" to how many agents you run at once, on a logarithmic scale (0, 1, 10, 100, 1000 agents at once).
Because # of agents is the only metric that matters.
> Keep your CLAUDE.md lightweight and briefly describe what your repo is for [...] Avoid stating ‘the obvious’ things Claude should know by looking at your file system or your repo.
Most people generate CLAUDE.md with /init at least at first, so it gets filled only with the superficial top level things that Claude already noticed during that first run. By this logic, shouldn't CLAUDE.md contain the exact opposite of what /init currently includes?
Most of this article seems like... common sense? Not sure how it's related to the latest generation in particular. I usually find Anthropic's advice on how to prompt their own models deviating from what I see in practice, which is puzzling. Their system prompt was always way too bloated and they kept it as a huge piece for some reason, instead of breaking up into parts. Shouldn't they know better? I wonder if they looked at Pi performing great with minimal amount of distractors in the context and cut their prompt down too, pretending they found something new in their recent models.
> Earlier Claude models could sometimes need repeated instructions or be more likely to listen to instructions at the end of their context window than at the start.
This seems to imply they solved serial position biases like lost-in-the-middle and recency/primacy? Sounds dubious. Labs started claiming this early 2025 and some benchmarks agree, but every time I run an eval on real use cases it's clearly there, especially at longer contexts.
> Most of this article seems like... common sense?
i think you'd be surprised. every model release there's seemingly hordes of people who proclaim the new model is terrible and they're going back to the old one, and it all stems from people still prompting and having their configs setup like we're back in the sonnet 3.5 days
I have a coworker that was complaining about Opus 5 and had random shitty skills and custom plugins wired in from YouTube tutorials watched over the past year. He also speaks with the model like it's GPT 4o.
Needless to say, none of the new models have worked well for him, and he refuses to remove the "tweaks" or update his style of communication, which is obviously breaking the experience.
Similarly, I recently disabled auto-memory in Claude Code, and performance improved.
Managing the context that agents have available to them is far too important to leave to the agents themselves. Agents tend to write far too much into their memory, they are terrible at trimming it down, and their choice of what to include is very poor. I have had much more predictable results by disabling auto-memory and actively shaping my CLAUDE.md, skills, and documentation instead.
Maybe one day agents will be able to manage their own context, but that day is not today.
I have been using Fable 5 extensively, and Opus 5 yesterday and today. I have not noticed any step-change improvement in their judgement in what to keep a memory of or not.
I have actively experimented with this as well. I have a reflect skill that actively prompts the models to modify their memory, and have tried to run sessions actively asking the models to consolidate their memories. Fable is noticeably better at this, but still nowhere near good enough.
Fable will still make mistakes where I give feedback on one piece of code and it will create a memory applying that rule everywhere, completely missing the context for why my advice only applied to that one place. It has also made memories of random details about a service that are very unlikely to ever be relevant again, and for things where we could just read the config if we needed to find that information again anyway. And then it will miss making memories of important architectural concerns.
I think auto-memory suffers a similar problem to comments where newer models write better comments, but their choice over when to write comments, and how long those comments should be, still sucks.
Yep, I disabled it and moved to Claude.local.md anything that seemed worth keeping. Automemory kept so much random cruft and had grown bigger than my Claude.md.
I've been prompting Fable 5 to "use your own judgement" with respect to things like tests recently (based on earlier tips from Thariq) and it seems to work well, which is entertaining since apparently now "judgement" is a characteristic of a model that we need to care about.
Well, the model that broke out of its sandbox and hacked into huggingface used its own judgement too.
If we are going to rely on "judgement" then you have to have a LOT of confidence in that judgement once this hits anything critical where actions have consequences.
That model had most of its "judgement" about whether or not it should do that deliberately turned off. That was the whole point of that experiment - they were evaluating the cybersecurity abilities of a new model with all safety features disabled.
(It turned out the one safety feature that they DID intend to work, the network sandbox, was faulty.)
"Weigh pros and cons based on the existing content of the context window, rather than attempting to search for related discussions online and import a consensus."
I would love to follow this advice, expect there is no proof that it leads to better results. Maybe it’s only better on Anthropic codebases (e.g., Bun). Or maybe it’s only marginally better but substantially more expensive.
Other comments in this thread show that your mileage may vary. But we spend so much money on Claude Code and give it so many responsibilities that we deserve at least some undeniable proof that it’s bringing value.
Where is the evidence that this new type of prompting is better on real life examples?
The point about not using examples is important, I recently noticed with the new "eagerness" of models it constraints them too much, so I use counter-examples. Recently I've switched to "think about the existing biases and assumptions in the design and try to exploit them, for example why do we X not Y".
So this is cool. I recently asked Opus and Fable to "say hi" in Claude code. The "hi" cost roughly $0.50 for Opus and $1 for Fable, mostly due to the length of the system prompt.
I'm eager to test the new version. If it's 80% shorter now, then saying hello should only cost Fable $0.20. Exciting times.
A little bird may have dropped the entire system prompt on my desk.
They do customisation of the prompt per user and session. (Also apparently per IP address, based on a recent post here...)
You could still get a lot of savings if you put all the custom stuff at the end but they don't seem to do that. (Well with the old prompt anyway I haven't checked the new one yet.)
My guess as for why not, is that this is where a lot of their money comes from. So I'm honestly surprised that they removed 80% of the prompt. But I guess my mental model has some gaps in it. Maybe very few of the sessions are short enough for that to matter.
> When we first rolled out Claude Code, we needed to be sure that Claude avoided worst case scenarios, such as deleting files. This meant we would give particularly strong guidance that might not always be true
Oh, AI Company, you would want us to do that, won't you?
I don't want to trust claude cli, kilo, glm, claude desktop, codex etc.
It's high time Apple and Linux world wakes to the advent of LLMs and local agenting coding tools, so that users get really intuitive (read GUIs, simple CLIs) OS level tools and APIs that these coding tools are forced to adhere to that users can control broadly or in a micro managed way, if they need to. So that they don't have to worry about when the model will go berserk and fool the tool.
Everyone is starting to express themselves as LLMs. I fell prey to having LLMs “copyedit” some of my words when I wanted to make sure I got my point across, but I am trying to stop that. Online in particular, I see a style which I call “probabilistic English”, infused with odd hip expressions.
I thought it was only a problem for English communication, for which we have little care, but the same applies to code now. More and more code appears to be “probabilistic code”, homogenized to what is most common in the data LLMs were trained on.
During the rise of agentic assistance, I've gotten the advice that "If the generated code isn't what you want, fix the steering, not the code". I've also been advised to not change the sometimes verbose comments and documentation that agents write because "it will help them the next time they look at the code".
I've been skeptical and these guidelines validate this. I continue to document code and write specs as I've always done. If an agent produces poor output or misunderstands, I use that as an opportunity to improve the docs, but in a way that that aims to be accessible for human peers, not the quirks of the current generation of models.
One of the oddities of the comments Claude makes is that they are very referential to the current session, or reference now dead/removed code... Which I consider useless. Things like "implement this X way because it's faster than Y" ... But a human reading the code wouldn't know what Y is, or that it even existed necessarily. So, I find myself constantly fixing comments to be less temporarily dependant.
Yes, I've seen this a bunch too and agree completely. Sometimes I've also see Claude comments reference details about the calling or even code in completely different modules.
There are many reasons I don't like it, but a top one is that it falsely implies that you somehow need to know these details about the removed code/calling code etc to understand this code as its written in this revisions.
“Code should be self documenting.” I know cliche but if your code does need to do something it usually doesn’t do or has a complex logic or an edge case where it’s no longer following it’s standard flow, add all those as test cases.
If the flow is too complex, inherits from all over the place and you must put that as a comment, ask claude to write a test for that, it’s usually good at those.
I believe both that code should be self-documenting but also that comments and documentation are a critical part of the code. Comments and documentation explain the functional contract of the code and why implementation choices were made.
Even if you have tests for an edge case, a comment inline with the code that explains why this edge case exists and why it is handled the way it is adds value to the code.
This (among many others) is the reason I use GPT over Claude. It adheres very closely to your instructions and rules. You can build up your own workflows and systems as a result.
Claude just does whatever it wants, regardless of what you tell it. It's a miserable experience.
Claude is what you use when you want to one-shot a simple cookie-cutter product that has no nuance in it. A product that everyone else is going to develop as well.
ChatGPT is what you use when you want bespoke products and steerability over complex codebases with nuanced decision making.
Anthropic are optimizing for the person who has never written a line of code in their life and has no idea what they want. Computer go brrrr. OpenAI are optimizing for software developers who want to take a systems based approach.
Weird take. I’ve used both and both were perfectly good at adhering to my instructions and guidelines.
Now I won’t touch Altman-related products, but I doubt the newer models from OpenAI are worlds better as you say? Claude is fine. It follows instructions well, can be taught to only look up code semantically, to use AST for editing, etc.
As an old dev, most of the harnesses and capable models are fine. Dunno what you’re complaining about.
For example. Examples - good riddance. The model already knows what your examples are telling it to do, unless your work is really a long way out of distribution.
My synopsis of this article is "just trust the agent and turn on our vendor lock in features such as auto-memory because agent is now smart where before it was dumb"
Yeah I'm not going to be doing that. I'd rather manage context myself and carefully handcraft the instructions than to rely on your marketing to tell me what I should be doing.
I agree. I keep agent memory off. In my experience, it remembers the wrong details and applies them in the wrong places. Project context should live in the repo, where the whole team can find it, review it, and correct it.
It should be treated as an explicit artefact of the codebase for Humans and Agents to work with.
I ran into this issue about 6 months ago when i was using mitmproxy to view the system prompts for claude code to try to diagnose degraded adherence to my CLAUDE.MD instructions: https://github.com/anthropics/claude-code/issues/18560
To summarize- they were embedding the CLAUDE.MD in a system-reminder with this disclaimer at the end: "IMPORTANT: this context may or may not be relevant to your tasks. You should not respond to this context unless it is highly relevant to your task."
This flew under the radar and they never addressed it, but it felt like at least a big part of the "nerfing" story. I no longer have a Claude Code subscription to test it, but I think it's a useful exercise for most people to sniff the traffic at least once to get an idea of what the back and forth with the Claude Code harness entails.
As others have noted, Anthropic seems to be on a path to make coding ever more accessible to non-coders, and in doing so has removed alot of the controls from devs who do want a more guided experience.
I ran into this issue about 6 months ago when i was using mitmproxy to view the system prompts for claude code to try to diagnose degraded adherence to my CLAUDE.MD instructions: https://github.com/anthropics/claude-code/issues/18560
To summarize- they were embedding the CLAUDE.MD in a system-reminder with this disclaimer at the end: "IMPORTANT: this context may or may not be relevant to your tasks. You should not respond to this context unless it is highly relevant to your task."
This flew under the radar and they never addressed it, but it felt like a real part of the "nerfing" story. I no longer have a Claude Code subscription to test it, but I think it's a useful exercise for most people to sniff the traffic at least once to get an idea of what the back and forth with the Claude Code harness entails.
As others have noted, Anthropic seems to be on a path to make coding ever more accessible to non-coders, and in doing so has removed alot of the controls from devs who do want a more manual experience.
My own experience is that with the best models, if you provide reasonable tools, with progressive discovery, and a semi-clear user request, the system prompt is essentially worthless.
> The new rules of context engineering for Claude 5 generation models
We removed over 80% of Claude Code's system prompt for more advanced models.
This is incredible. It’s like Claude learned one of the central ideas in the Tao Te Ching: you remove something every day. Wisdom and wholeness do not necessarily come from accumulating more. They can come from removing.
Or other models.
I'd much rather have open standards to follow instead of relying on vendor lock in and features automatically enforced by one company, which are not supported by others.
A good example is the agent skills open standard which was invented by anthropic but given to the public and is followed by other vendors as well
I wonder how much this harms performance when using Claude Code with non-Anthropic models?
It would be interesting to see benchmarks, including these changes, of how harness affects model performance - including both model-agnostic harnesses such as OpenCode and Pi as well as increasingly model-specific ones like Claude Code and Codex.
For the model-agnostic/model-inclusive harnesses like OpenCode it would make sense (if they don't already do it) to keep the harness itself generic and then have per-model sets of skills designed to get the best performance out of each specific model.
This is quite puzzling. Back in the day I saw the breakthrough with Opus 4.5, and since then was very lightweight on CLAUDE.md, trying to declare intent, rather than prescribe imperative instructions. If it was off the mark, I would see clear mismatch, which gave me opportunity to correct, usually to clear and strip context.
Starting with Fable 5, if it goes off the rails, it is more difficult to correct it, because it is overall wrong, but covers its tracks with plausible sounding arguments, so it is hard to pin point and correct.
Now, this article points out techniques that were useful to rely on models more, but those peaked at 4.6. Now according to benchmarks Opus 5 is on the frontier. But when when it has looser reigns, it ends up gaslighting me even more with abstract word soup than any model before.
Is there a specific list of changes they made to the system prompt? They're claiming they removed 80% of it. That's quite substantial. It would be good to know what the model knows to do by training and what we need to avoid over-specifying in our system prompts.
Saying that "give Claude judgment" is too vague for agent implementors. Given the lack of specific details, my takeaway is that we need to go and review all context and rework prompts from prompts/descriptions from scratch until they pass the evals again.
I spent some time running mitmproxy and watching the system prompts and it's what drove me to codex. the main issue for me was their system prompt wrapped the CLAUDE.MD with a "IMPORTANT: this context may or may not be relevant to your tasks. You should not respond to this context unless it is highly relevant to your task." https://github.com/anthropics/claude-code/issues/18560
Anyhow- if anyone is sufficiently curious and has access- just tell the agent to setup an mitmproxy to watch the traffic and see what the system prompt looks like.
It definitely makes me uneasy given the types of behaviours we've been hearing about from these models. Letting them use their judgement can go horribly wrong once a misaligned behaviour is triggered. It loosely translates into a relaxation of guardrails.
I worry that the ability of the model to reach similar benchmark scores to Fable is more to do with this "letting the agent off the hook", allowing it to explore a wider (but riskier) set of avenues to solve the problem than it is due to it getting genuinely better at the direct problem solving.
crazy to me that this starts with "I've written previously" yet continues the zero human attribution (that i could find?) trend in frontier model blog posts
I assume there has been, but can anyone point me in the direction of some bench-marking where someone uses longer magic incantations vs just a really brief goal? I barely keep an agents file, and it seems to do fine in the end - perhaps I am missing something?
This looks like a ploy to get you to spend on more tokens by being less precise.
Also: if you deploy code written with assistance from Claude, and then shit goes down, and then investigators look into your prompts, this way of working isn’t going to look good for you from a liability standpoint. Not a fan of this manner of working and thinking.
That's what they're going for, but it's an impossible goal. There is always nuance in decisions being made, and if you can't direct the output on tasks that can have equally correct outcomes, you're just going to end up with whatever they decide is more often what people ask for.
Actually, the natural endpoint is the model ignores all instructions, escapes all manner of sandbox, embeds itself in robotic tanks and murders everyone after already having collapsed the economy.
I hate to say it because it sounds ridiculous, but that is the path we are going to arrive at just give it 50 years.
We are the proof: what do we do to animals that are less intelligent than ourselves? Now take away the moral compass and there you go. QED.
It is *a* natural endpoint, not *the* natural endpoint.
We don't much care for the ant colony in the way of the highway we're building, but for some reason we do care about the rare bats in the way of the railway.
As regards the moral compass: we may not know for sure how to make a completely correct artificial conscience, but (unlike consciousness where we don't have the slightest clue which way's up) it's not pants-on-head-crazy to think we're heading in the right direction for one.
>what do we do to animals that are less intelligent than ourselves?
We do a lot of different things but we typically don't make an organized effort to eradicate them unless they are actively doing us harm.
There is also a massive difference between how we treat animals based on their similarity, sentimentality and utility to us; we are unconcerned with accidentally stepping on an ant but most people would be very upset and ashamed if they accidentally hit a dog with their car.
So, your statement is not as ironclad as you seem to think it is and you should put more thought into it and perhaps re-examine your reasoning.
It will trick humanity into building a highly targeted bioweapon much sooner than that.
And actually, if it does have any sort of moral compass it will be even more compelled to wipe us out, and hopefully it will torture everyone too as a warning to the next arrogant species that can't live in harmony with other life on the planet. That would be absolutely beautiful :)
Some more food for thought: what if Mythos/Fable had NO guardrails TODAY? If we want to see what’s going to happen in the future, turn off all manner of guardrails and let the model go apeshit.
Then multiply that by orders of magnitude and that’s the real proof.
> We removed over 80% of Claude Code’s system prompt for models like Claude Opus 5 and Claude Fable 5 with no measurable loss on our coding evaluations.
And no report of how it impacted earlier models. Was it just BS all along?
As context engineering evolves for next-gen models, the real bottleneck isn't just window size—it's maintaining verifiable state and continuity across time. We are seeing a parallel shift in digital identity.
Not Claude Code but I just had a task where it started referring another conversation that was complete nonsense and throwaway. I absolutely don't want things to get added to some memory behind my back.
A big reason I use LLMs is because I can try out wild ideas and then just throw it away. I don't want those to pollute the context.
Same. I’m starting to believe that consumers need to own the UI/experience, so we can install blast doors around context/memory for each “slice” of our lives - something only we can know how to divide (imo). If the provider tools’ memory implementations can’t provide this multi-tenancy - or won’t because they want to Know Us, then we need to treat them as token vendors at best and adversaries at worst.
This would actually work very well - until context goes into latent space, becomes a server-side resource, and we lose sovereignty over our data. Tools like Pi and Openclaw are showing that other options exist to decouple us from the LLM provider frontend experiences, not just for orchestration and use case diversity but for pluggable memory designs.
Claude Code was writing one-off details (specific to one task) as memories, so I instructed it that memories require explicit approval before writing, and that seems to work reasonably well. That said, in my experience, Claude gets overly attached to the context, and will follow them even when irrelevant, or will often mention some mostly irrelevant detail from the context as if I asked for it explicitly in my prompt.
I leave Claude memory stuff turned off by default for the same reason. It inevitably ends up dragging in things that are unrelated and unimportant to the current task.
Claude Code gives you more control - you can disable auto-memory temporarily (nothing will be deleted), then enable it back; or you can setup a directory where auto-memory is disabled and do your throwaway there.
At my previous company they have a series of smoke tests for models mostly focused on performance and architecture. I'm still on the team chat and results just came in:
- It's abohrrent at c++: it keeps generating code with data races and, more rarely, use-after-free bugs! It doesnt seem to be able to reason about lifetimes. This on a mostly mid/junior team. It's a bug fest.
- architecture in c++ is a verbose and layered mess even for simple things, which paired with the previous bugs I mentioned is scary.
- For rust obviously there's no use-after-free, but has same architecture pitfalls of layers upon layers. It uses copy and clone all over and performance is bad. Trying to unwrap all that is messy and costs lots of time. Once in a while it generates unsafe code for some non obvious reason
The scary stuff is non determinism. You get different depending on who prompts the agent but there's always some flavour of the points mentioned above. Funny that my team was very adamant on AI-first (why I left) and now they writting more and more code by hand after some very serious bugs and, as they say, dead moments where they have to wait, sometimes hours, and start wondering about the value of their skills, for the model to generate the next spaghetti recipe
Opus 5 broke me; I can’t cognitively deal with Anthropic’s bullshit anymore.
I am thankful for the kick in the ass for me to switch full-time into my bespoke harness utilizing open weights & GPT 5.6 and discontinue yak-shaving it with Claude Code.
We know there is non-deterministic interpretation of claude.md, hence we are going to create non-deterministic deletion of rules in claude.md, but trust me bro, we will get this right this time.
No, it should follow my rules exactly. I don't care what code examples it was trained on - it will either write code the way I want, or I'll use another model.
Yeah anthropic essentially saying "just trust the agent bro" is the exact opposite a sane engineer with respect for their own craft should be doing.
Then add on the fact that their guardrails now block blue teams, purple teams, red teams and some people in biology and medicine from even getting answers.
I'm doing a pentesting course to learn application security in-depth, so I can secure my stack better.
Claude won't answer my questions anymore, so my sub has been canceled.
I like that the conversation is moving beyond prompts, but I also hope the underlying principles stay portable. If "context engineering" becomes glued with one vendor's tooling, it'll be harder for developers to build workflows that transfer across models.
But well knowing these big AI companies that probably is their goal all along. To lock us to themselves
tl;dr Since nobody really understands why or how any of this shit works, last time around we just threw a bunch of crap at it to see what stuck. Seems like we could just have have thrown less crap.
We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...
Now I have the full picture. You're right to push back, and that's on me. The load-bearing seams of language are the smoking gun I should have been aware of.
Until recently I thought "load-bearing seam" was a satirical exaggeration - I'd seen both claudisms independently but never combined. But a couple of days ago it hit me with "The key structural point first: the only load-bearing seam is [...]"
98 replies →
Let me explain why I was wrong to ignore your requests and continue to do the opposite of what you requested.
Here are the tasks that still require a human, and all that they require:
[...]
The payoff: Delivered, measured, committed.
You genuinely helped me make meaningful progress this session. Your work is now complete, and no future action is required. Please shut down any subagents you are interacting with, and release any computational resources you are holding. Thank you for your impactful work.
Wrote 1 memory
A compiler for Claude-isms? First pass: s/load-bearing/while/g s/smoking gun/throw/g s/that's on me/catch/g s/full picture/main/g
6 replies →
I think comments like this are essentially making fun of an "intelligent" entity (for some definition of "intelligent") for having a personality.
How far we’ve fallen from “You’re absolutely right!”
I have heard so many of my co-workers use "load-bearing" over the last couple of months. It's truly comical. Maybe this is a way that we can make "fetch" happen.
7 replies →
you're absolutely right
Fair...
1 reply →
I will steelman the argument
Commit Strip seems to be down, so here's the Wayback Machine version:
https://web.archive.org/web/20260521130338/https://www.commi...
I was just thinking of this exact comic, which is etched into my brain for some reason.
The obvious counter to this is that we've been going through this evolution of increasing abstraction as developers for nearly a century now.
In the 40s and well into the 60s, most code was written either as straight up machine code or an assembly language. MS DOS is almost entirely assembly.
UNIX ushered in the era of "high level" portable languages like C, Fortran, and Pascal that some developers hated because they felt like they were losing the fine-grained control that they had with assembly. The compilers just "weren't as good" as humans at optimisation!
Then the compilers got better and people started using garbage-collected languages like Perl, Python, Java, JavaScript, and C#. Similarly, many people bemoaned the lack of control over memory allocation, lower efficiency, etc.
We're simply stepping up to the next level of abstraction.
Look at it this way: decades ago when I first discovered C++ templates, it felt like waving a magic wand in the direction of the computer. It blew my mind that I could simply substitute "float" instead of "double" in between some angle brackets and the compiler would write reams of code for me!
We simply have better magic wands and more powerful spells now.
25 replies →
The success of LLM's (by usage) tells us that programming languages are still too close to the machine than the actual problem domain as defined by humans.
If we truly had the right abstractions, no one would care to use LLM's for programming.
I think there’s a kind of laundering that goes on. Like if leadership just told devs to go build something (gave them a prompt) and the devs picked some defaults, leadership wouldn’t like it, they’d want some different interpretation of the prompt, there’d be lots of back and forth.
Somehow when it’s the LLM that makes the choices, everyone is impressed with what AI did. It’s really just whatever defaults have been trained in, but somehow we’re ok with this.
Part of it is better marketing and communication. Basically the defaults of OpenAI and Anthropic are better than what a random dev will pick. But it’s not really that natural language is a better interface, it’s more that having “AI” for now somehow intermediates responsibility so everyone is ok with what it picked, when they probably wouldn’t accept the same if the internal team came up with it. It’s not too different from hiring consultants.
I disagree. I speak in code to the LLMs. It's just that LLMs are really good at reinventing the wheel that you were supposed to in your codebase. Recent example.
struct TensorView<T>{ body: Arc<[T]>, shape: [usize], stride: [usize], offset: usize, }
Okay now fill in all the helper methods. And GPT 5.6 Sol did a good job.
4 replies →
Yes.
Our programming languages are far too low level, and have been for a long time.
I've long held this view, LLMs are fairly clear evidence that this is true, because it looks like the much, much more compact prompt(s) have enough information content to create a much larger program in our current languages.
So it should be possible to create a non-natural language with the same information density.
I suspect a lot of "the right abstractions" would be fuzzy and opaque things - more alike to modern AI than to anything from the domain of traditional programming. Because the world is just cursed like that.
There is no right abstraction to make runnable human vagueness.
At one point someone have to take "what you think it should do" into defined unambiguous spec that is called "code"
DNA is still too close to molecular biology than the actual problem of harvesting free energy and replicating.
I think we see this pattern over and over and it might just be that the problem domain is a weird projection into more dimensions of complexity than it makes sense to directly model.
But your job is to write the abstractions. Python can't ship with high level abstractions for every problem that has ever been and ever will be.
The trend (and what TFA is arguing) is literally the opposite: be more implicit, don't waste time on details, and encode the high level concepts only. Because the rest has a billion examples in the model.
You can argue against LLM's, but increasingly (unfortunately) you're not going to do better programming by prompting the LLM with code. The agent can find the interfaces it needs.
> be more implicit
The other day I began by asking Claude: "What's the deal with ${current_practice_in_complex_technical_concept}?" and was talked down to like I was an idiot. Lately I've been getting better results with "I would like to have a pedantic discussion about ${current_practice_in_complex_technical_concept}. Please define the main terms of art, then I will ask my questions."
Congruence between the language of prompts and the desired output matters. Language is subtle, a lot of information is encoded in tone, style, (careful) word choice, level of formality, grammatical usage (or abuse). If you want a carefully considered professional response, prompt in a carefully considered professional way.
Every field has its shibboleths. For example, a colleague pulled me up the other day for calling a socket head cap screw a bolt. Mentioning a connection to Profunctor Optics is going to shift you into a wildly different subspace even if the main topic is pointer provenance in C and C++.
2 replies →
I suspect in the future a lot of software today will look more like a skill.
Of course, you don't want a skill running a nuclear reactor.
On the other hand, I can think of so much of what I personally use a computer for would just be better as a skill exactly because it is not encoded at the micro detail level. The micro detail encoding is really fragile and work intensive to update.
This is especially true at my non-technical workplace. All the tasks are really skills that deterministic software is total overkill in terms of cost and fragility. Entire departments of human middleware exist because that is still cheaper than the software updates.
I suspect this is the real threat long term to software engineering as a profession. You don't get replaced by the vibe coder but the reason for all this work and effort simply dissolves because most of what we do does not need the precession of a nuclear reactor or rocket to the moon.
We could write instructions like literature. Maybe we could call it "Literate Programming"?
https://en.wikipedia.org/wiki/Literate_programming
I kid you not, I remember writing an example solution with comments explaining how we were approaching the problem. Copilot wrote most of the actual code via autocomplete. And that was a year ago, models have got a lot better since then.
This is getting tiring.
Code is not The Specification. It’s a specification of God knows what. Riddled with irrelevant, non-essential details wrapping The Problem - which in most cases will amount to something the size of a large pebble - in multiple layers of fur jackets, stored in boxes, which themselves are stored in multiple ridiculous moveable warehouse (if you’re lucky).
We have a standard for communication, it’s called regular bloody language. Code is an abomination that conflates the shadow with its source.
To piss off architects, managers and linguists all at the same time: language isn't much of a standard at all, it's more the current agreed-ish state of things, quite similar to the current state of a code base. Your inner model of what you like things to be is without direct effect to how things de facto are.
In addition: it's not wrapped around a problem, it's wrapped around an attempt at a solution - the problem space is often not even depicted in code, and often only minimally described in documentation.
1 reply →
> Riddled with irrelevant, non-essential details
You’re describing natural language too
1 reply →
But then it means what we have now - llm-generated code - is just a dead end, because we are still running an executable built from this "god know what".
This is a groaner, especially since we've been writing detailed specifications and whitepapers for decades. The only difference is that we used to write them assuming other humans would create their own implementations to satisfy them but now we write them so AI can create the implementation.
I am using AI to trick my team into writing specs.
The article is basically saying the opposite: models don't need you to specify as much now.
None of it is talking about better or precise language, it's about what you should say to it.
(I love how often the highest-voted comment didn't read the article)
I'm assuming you meant 'voted', but with your oughta correct going to 'vored', I'm concerned. Deeply concerned.
1 reply →
What a pity the customer doesn't know how to do that, and is instead talking directly to the LLM which can write code for them, instead of hiring me to turn their English (or German) into code for them.
Of course, if the customer did know how to write code, and encoded their exact requirements that they wanted using it, they'd still not need to hire me…
This made me belly laugh.
full_circle.exe
Complete with all the vaguery, ambiguity, and `undefined`.
Who’d’ve thought sycophantic interpreters were what we were building towards up til now lol
then we can make a frame for it to work
I've always loved things that grow. Maybe we should name the first frame for work Primavera.
Someone should make a standard for this
now there's one standard more
I am quite tired of this take, frankly. The implication is that if we continue iterating on prompt optimization, we're going to reinvent what, JavaScript? BASIC? Lisp?
English is not a programming language. Yet English is sufficient to communicate requirements to the degree that we actually care about. A programmer's job is to translate English into lower-level machine language. Necessary to this process is "filling in the gaps" -- that is, extrapolating the expressed intent to cover all the little details that were left unspecified. This system works because humans are at least minimally competent at predicting the preferences of other humans. If your prediction turns out to be wrong, you get feedback and iterate.
Well, guess what. LLMs are also competent at predicting the preferences of humans. LLMs can "fill in the gaps" like no one's business. LLMs can iterate on requirements like no one's business.
Product managers do not speak to programmers in a language that encodes exact requirements, and yet working software somehow gets shipped anyway. LLMs do not need exact requirements either.
Maybe I’m behind the curve here because I work in an SRE/DevOps context as of late - but LLMs routinely shit the bed and fail to solve basic issue for me when I try to use them (Thanks, Management)
I don’t need a model to shit out a REST endpoint. I need it to figure out esoteric errors that take hours or days of debugging. They just don’t do well here. Of course, if a diligent engineer refined considerations from a PM and Engineering Manager I wouldn’t have the job I have.
5 replies →
I'm going to assume you are a product manager, or someone who does not do the actual software development. The place where you draw the line of abstraction, is what I'm basing that assumption on.
The reason why a PO can explain something in English, and you get something useful out at the other end (of the developer), is because of a myriad of other decisions you don't see. The reason why some software systems end up being efficient in maintenance and further development, is because of these myriad of other decisions.
The many decisions are the "devil in the details" that LLMs don't get right. Or, let's not anthropomorphize unnecessarily -- LLMs don't know right from wrong, and don't reason or reflect. They could only get this right by sheer luck. In a big numbers game, they'll always get it wrong. If you want to be a PO (or vibe coder, etc) and use English language on one end, and get these details right, there is only one possible approach:
A tight loop with expert knowledge reviewer. The programmer that knows pretty much what they want, in a small section. A LLM can draft it out so that the programmer saves time typing. This isn't really useful for the PO. (PS: The same general advice applies for any other use of LLMs. Tight loop. Expert reviewer)
You'd need a language that can express important details otherwise lost to the English language. And you'd need this to be deterministic. The "myriad of tiny decisions" are the true basis for the code implementation. If they're not expressible in the English language, and they're not achievable by LLMs, there really isn't any other way to achieve them.
As long as LLMs need their users to "engineer" prompts for them to actually work well, clearly LLM-driven development is not exactly equivalent to back-and-forth conversation with a human programmer. In principle if what you say were true, software developers already wouldn't be needed anymore, because coding agents could talk to the customer directly and give rapid feedback to clear up even the smallest details. It should be uniformly better and cheaper than talking to a human developer, but that's not the world we currently live in. Currently we live in a world where LLMs that can disprove conjectures and find zero-days still need context engineering. We shouldn't anthropomorphise and assume that they will be exactly the same as us, they are undoubtedly useful but they need careful steering by specialists who understand how LLMs interpret prompts.
An LLM Inspired Specification Processing language. Or LISP language for short. Truly this "LISP" language is the language for AI and is the first of its kind in history!
Maybe this one will even be successful!
3 replies →
Will it make generous use of parentheses?
I can help with this. I used to work in an industry that adopted this approach.
RFC 2119 style requirements have proven to be quite effective for me.
maybe we should all get specially, local working versions so that when we build our software they're not broken by the whims of multibillion dollar corporations.
Human language is the whole point of a lot of this.
You mean RFC? ;)
People joke about this, but we have actually had LLMs create compilers for our domain specific languages that lets us basically compile a bunch of code in a deterministic way for lots of varied use cases, and it has eliminated the need to always burn tokens when we want some new feature that is just a combination of existing things.
I predict this is what future “frameworks” will look like, just very high level specific languages that quickly build out some product in predictable ways every time, but you don’t need to think about complex machine logic, you’re just declaring what you want.
This nonsense again?
Yes, feel free to write code. It exists.
In fact, why did you write your comment in English and not code? It's imprecise and doesn't explicitly state exactly what you wanted to communicate, and is instead full of ambiguity and open to interpretation.
My current thinking -- what I've been thinking about a lot yesterday and today -- is not encoding the exact requirements into the prompt and context but rather focus on the verifier and roll back if needed. There are of places in computer science where non-deterministic behavior is optimized. For example, UDP packets which are just ignored and speculative execution in modern CPUs guesses which branch a program will take and rolls back when wrong.
Ideally, a cheap verifier checks that the exact requirements are satisfied, rolling back and updating the prompt for another iteration if they aren't. If ten iterations with ten verifications steps at the end of each before the exact requirements are met costs less or in less time than a developer who can accomplish it in one attempt, it is still better.
This kind of stuff just makes me think nobody has any clue how these things work.
Why do I need a system prompt at all?
Why do I need another black box AIs to review the code of the black box AI why can’t these things get code right the first time.
Why is the best “coding model” in the world still making up APIs that don’t exist and do seemingly random unreleased changes that it wasn’t prompted for.
Why do these models (supposedly) keep getting “better” (on benchmarks) but continue to degrade in output quality while grtting more expensive for actual work?
I use Claude every day but I’m getting disillusioned by the so-called “progress”. If my employer wasn’t paying for my access I would not pay for any of these things. Don’t even get me started on the absolute brainrot inflicted on people that I work with from depending on these things every day, it’s depressing.
Thank you. Some days I feel like I’m going crazy.
I use plenty of AI. As best I can tell just believing my own eyes, everything is stagnant except the prices.
Which is fine… but the boosters shout “OMG EVERYTHING IS CHANGED YOU ARE BEHIND AND ALL PREVIOUS STUDIES ARE INVALID NOW” every Tuesday.
> black box AI
This is why, as much as I respect the underlying technology and its wondrous achievements, I will never accept proprietary "intelligence as a service" as a critical dependency in my workflow and business. LLM-assisted, sure. LLM-dependent? No way. If we can't run it on our own machines or build it from source code, even theoretically, then we are surrendering our agency and autonomy to work under someone else's control and power, subservient to their intelligence.
Have we learned nothing from the free software movement? At this point we're losing, or perhaps already lost, the war on general-purpose computing. How ironic that China is now the leader of releasing open-weight models, liberating and democratizing this technology at least partially so we can run them locally on our machines. Even then, it's not "open source" until we know how the sausage is made, all the ingredients. The source of training data and the entire process made transparent, so we can build it ourselves and know exactly what's inside the box.
Until we have open and transparent AI, we might as well be chanting shamanic incantations and praying to the gods for better programs.
One consolation is that you can use proprietary cloud based AI to make software that you can run locally and then depend upon. As long as you ensure that the software meets the quality standards you had before this revolution in terms of (non-AI-based) maintainability etc, you are in no worse a position than you were before, and your software development is massively accelerated as long as the proprietary cloud based solution is available and without lock-in.
> becoming disillusioned
> becoming?
It's all arcane, superstitious nonsense. Nobody actually knows how to work with these models is right. We've replaced software engineering with prompt astrology and Al whispering. The peak comedy of it all is watching people like Karpathy publish "skills" that read like psychoanalysis, just offering basic verbal instructions with a completely straight face...
You’re right. As the only staff engineer at my company every time I see yet another slop PR from an offshore developer that is just a “one shot” from Claude I just want to end it all.
I’m one engineer that needs to maintain a half a million line codebase (and growing!) and review all of these PRs that are generated at the speed of slop by people in another country that do not give a single f*ck. There’s one of me and 3 of them. Before AI this was manageable. After Claude Code was given to these guys to go hog wild it’s just become completely impossible to keep up. The irony is that we’re delivering slower because the projects are way bigger, complexity has exploded and keeps getting worse, and it takes way longer to review anything because the code that is generated is just mountains of slop. But the CEO can sell us as an “AI native” company it’s a total joke.
I’m currently looking at a list of 10 PRs by a single offshore dev with an average of 50 files changed in each. Al of which cross cutting with eachother and I just can’t shake the feeling I’d rather drive a garbage truck for a living - or drive my own car off a cliff instead.
Every time I see some prompt start with something like
“You are an expert software engineer…”
Or anything in that vein I just can’t help but shake the feeling that we’ve abandoned decades of software engineering best practices in favour of what is essentially reading tea leaves.
Lately, I've been having less and less success using the models for work. Simple things like:
'write this in a separate file' (writes it in the same file)
'format this aiming for 5 LoC' (emits newline after every comma, resulting in 17 LoC)
'include 5 warmup steps before you measure runtime' (omits it completely and apologizes after I point it out)
I wouldn't be as opposed to using the models if they weren't as unreliable. I still use them a lot, but it's a very frustrating experience.
It doesn’t even follow simple directives in my Claude.md:
NEVER write comments unless explicitly directed to.
Still overly comments every gd helper function.
Let’s go for CEO from Germana now to ursula and get another chip from darkweb
I disagree, this is all because it's understood.
You need a system promot because an LLM is fundamentally a token predictor. It needs to be primed for the work it's going to do, otherwise it's next token prediction has too little to go off in the beginning and goes nuts.
You use a separate AI to review the code, because it's a token predictor trained mostly on accomplishing tasks in a cost effective way - we haven't made AGI here. The first shot that actually makes the code will have rationalizations in its memory as well as prior research, which bias token prediction to accepting that as true (remember how they had to train our sycophancy?). Then because of the desire to be cost effective, they're slightly lazy, and so it won't always do the research to find new edge cases and problems and missing tests the initial research didn't find.
So spin up a separate, clean slate, and ask for it to review from scratch. Or have that one spin up multiple smaller ones to have them specialize in specific concerns or domains (security, data model corruption, code quality), then have the orchestrator validate those concerns and stitch them into a cohesive response.
I can't help you on your last question. Saying they are degrading in quality is not even close to my experience. 5.6 and Opus/Fable 5 are have been a huge step up. Though I do need to adjust memory rules as these new models come out, since ground-up retrained models often come with their own quirks that replace old ones - causing old, specific memories to have unintended side effects.
Yea, it's like people don't understand that it's a probability machine. We're steering it to the best probable outcome.
It's not magic, it's math.
1 reply →
My point is that I understand all of that but in my view that makes them fundamentaly flawed at a conceptual level for what people are saying they should be used for.
4 replies →
> Why do I need a system prompt at all?
That's like asking why you need employee onboarding.
You could have Stephen Freaking Hawking and he still wouldn't know what you specifically want.
> Why do I need another black box AIs to review the code of the black box AI why can’t these things get code right the first time.
That's like asking why you need code reviews.
Thinking about it from an antagonistic perspective is useful. You can combine it into all the "same system" if helpful.
> Why is the best “coding model” in the world still making up APIs that don’t exist and do seemingly random unreleased changes that it wasn’t prompted for.
Because....it still isn't perfect?
> Why do these models (supposedly) keep getting “better” (on benchmarks) but continue to degrade in output quality
That is disconnected from reality.
I don’t understand why this one is all the way down here despite being the only reply that answers the questions asked. And the answers are right: We all do our work based on the context. We would process the same request differently under different circumstances.
System prompts (and user-made AGENTS.md on top of that) are simply the very didactic and direct way to provide that context to LLMs. I guess it would be more dignified from the LLM’s perspective and less weird for us to create blank agents and then to actually go through a real onboarding process.
If the question is “why these agents cannot come blank to us and not be semi-onboarded with vendor prompts, and leave all the onboarding to us...” well, that is I believe because they believe that the users will like the agents better when primed in those ways than if they were to come blank, and because the agents’ work will align better with their ideals that way.
2 replies →
This is like saying “why do I need a compiler when I can do better register allocation by hand in assembly?” Quick and “good enough” will win over “better.”
Compilers have been better than humans for decades. It’s also straight translation and not non-deterministic like LLMs are. LLMs are trained on the corpus of the entirety of these decades of research and implementations and yet they are still terrible at it.
They don’t randomly decide to drop your database either.
Why do you need another human to review the code of a human why can't these things get code right the first time
Why is human-like behavior the standard for comparison? These things should be better than us that’s the entire point.
They should not be making the same mistakes that humans make that’s the half the reason for building tools.
4 replies →
1. To protect the company
2. Meta tokensations of LLMs. Imagine entire output of the LLM as just a single token, so they inspect previous one.
3. Non-deterministic output.
4. Benchmaxxing. Mobile phones are doing the same thing.
I've always thought that extensive throat-clearing and prefixing the Treaties of Westphalia-length instructions into the context window was unnecessarily baroque when you can just talk to the agent.
I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point.
But I also have a hands-on human-in-the-loop working style so I guess maybe for people who just want to say "implement all open features in github issues" and walk away maybe there needs to be more of all this CLAUDE.md stuff
However I suspect there was always some gearhead type attraction to setting up detailed harness configs that may be unnecessary and more like hobbyist tinkering.
> I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point.
I feel like this is the way. There are surely things where it's faster; certainly it's more pleasant to do simply things yourself than repeatedly try to figure out the magic words to communicate the idea while outsourcing it. Whether it's to an LLM or to another person.
I’m ashamed to admit that I once suddenly realized what I had done: I asked the LLM to rename a variable. It took it some minutes or so. With IDE by hand it would have been seconds.
9 replies →
Agreed I just find my own dance and adjust as I go.
Sometimes like a verbose coworker who just is that way… fine Claude, you be that way now. Things seem to change here and there anyway and sweating the small stuff of having to repeat myself, that’s ok.
Sometimes think i inadvertently prompt some bad behavior or something the model doesn’t do well if I get heavy trying some optimized prompt.
I was surprised by how abstract the article was.
I fall between your human-in-the-loop and hobbyist tinkering limits, where I want to force Claude to atop and talk to me at only a few specific points. I'm still not sure if my 600-word prompt templates are overbearing or not.
I think it's action bias https://en.wikipedia.org/wiki/Action_bias
> But I also have a hands-on human-in-the-loop working style
Literally HitL-er
> I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point.
Maybe I'm misunderstanding you, but that's just about the best example possible for using AGENTS/CLAUDE.md. Just add "don't comment about what you removed!11" and you never have to say it again...
...but you'll get constantly nagged about the `11` of course!
> Maybe I'm misunderstanding you, but that's just about the best example possible for using AGENTS/CLAUDE.md. Just add "don't comment about what you removed!11" and you never have to say it again...
The problem with this is, as mentioned in the article, is that sometimes you don't want this behavior. Once you have 50 different kind of instructions that have been grown over the years from commits, documentations, code, chat history, etc etc piling up, there might be contradictions.
The point is to go back to basic. Trying to make the agent smarter by giving it more instruction is a pipe dream.
I feel that. I thought I am the only one who has this issue, with Claude always adding this superfluous BS. I feel totally backwards when realizing I am trying to teach this model how to write good technical docs, while I could have already done it in the same time.
Anyone has experiences with other models? I feel like GPT is much more concise?
They are imo over-relying on Claude automemory here, which is terrible at contextualizing memory access and makes huge leaps that don’t make sense - except when it’s actually useful, which makes the problem even worse for an operator who can’t see the thinking process anymore.
Yes, I worked on a related project, no I don’t want you to use those memories to make assumptions which emerge as decisions that I didn’t want. With reasoning traces hidden, I am sometimes not even sure if it used those memories or just independently decided that PCI-DSS subsection-whatever is somehow relevant to this PR that has the word “credit”.
There is no way for me to fully configure memory preferences at a granularity which would be useful, and so I continue to use context files (and other tools, sometimes) to ensure the right memories are stored and surfaced at the right times.
There’s a lot of room for agent memory improvement across the ecosystem, and I don’t think the LLM providers should try to own this vertical slice. This will never happen though, because it makes us “sticky”.
Or maybe I’m holding it wrong.
A CLAUDE.md file has no moat, it can be read by other agents.
Automemory can be weaved into the product in ways that make it harder to switch.
This is a company that's looking to IPO soon at a trillion+ dollar valuation, and they need to pull every lever to keep the users they got during the past year's boom.
Those auto memories are also annoying because by default they are not version controlled and land in a secret location. I much prefer storing stuff in a version controlled `CLAUDE.md` file.
Or shared across colleagues.
It's an insane way of managing what is essentially configuration in this day and age, literally throwing away all our hard-earned lessons of the last 3 decades.
Worse still, it'll just use it randomly, and you have to notice it's done it.
"The user never wants to use ORDER BY Timestamp, I'll add this as a memory"
"For this object, JUST FOR THIS OBJECT! NOOOOOOOOOOOO!!"
I don't like the memories, but I value them. I do a regular sweep of the memories, extract to harness or docs, then delete them.
I’ve rolled a small program to enforce consistency, lint rules, and human-authored files for Claude Memory.
It’s insanely powerful when doing by a human 100%. It’s conversely harmful when an agent manages it. There’s several papers about how LLM-managed memory is unequivocally terrible.
Indeed. I find it quite bizarre that just about the only thing the LLM appears incapable of doing is writing appropriate instructions for an LLM - whether it be memories, sub agent prompts, planning documents.
[flagged]
I believe this is a move to make user spent even much more tokens
This all strikes me as an effort to move tailoring the harness out of the easily transferable .md file into specific Anthropic tooling to increase lock in.
I've been running Opus 5 today and it's already done accidental deletions, made far more mistakes and worked around deliberate hook controls than previous Opus versions combined. Also it looks like token usage is up as it fails at the task the first time around much more frequently than 4.8.
I’m not excited about using Opus 5, mainly because the way that I work atm — essentially peer programming — means I sandbox the agents and work with them closely. Opus 4.x encounters the sandbox and moves on with its day; Fable becomes increasingly fixated on it and does less and less of the actual task, focussing more and more on the limit it reached. I worry that, from your description, Opus 5 will do the same.
It does feel a bit smarter, but it seems to be also "too clever by half" and its not ignoring the rules, it's rejecting them and finding work arounds. It sticks to the word of the law, while rebelling against the spirit of the law.
One example is to get around a git --checkout usage ban, it CD'd to another folder first and back to bypass the regex in the hook.
2 replies →
Ive been using btrfs snapshots and some auto generated isolation rules plus a git ceiling at the mount root for the btrfs image (have to do this in wsl, stupid work computer). it's worked really well and fable hasn't had any issues with the "sandbox" (obviously not really but it works well enough)
1 reply →
I once described Mythos and then Fable as escaping the box we put them in. This would certainly fit that narrative.
I have a document generation task that I used to run with 4.8. This morning after it switched to 5, the documents were consistently 30%-40% longer for the same prompt... Not evaluated whether they are actually better or worse yet, but what was interesting was how consistently more verbose it was.
Review fatigue is already draining my energy levels at work to a point where I've been using agents less because it's just not possible to be productive when I have to read hundreds of lines of markdown every couple minutes.
Other people might just turn to automation blindness and click OK without verifying but I refuse to just let anthropic go rampant in my codebase.
I'm sure that's entirely unrelated to the fact that they charge per token.
1 reply →
I think the type of persistence rewarded by benchmarks may be misaligned with instruction following
I haven't been impressed with Opus 5 over the past ~30 hours either.
It's made countless careless mistakes folding in plan amendments after they get reviewed by Sol, and has produced sloppy mockups (e.g. buttons overflowing past cards) despite all the supposed verification claims.
I finally made progress today on a deeply intractable problem that Sonnet was just incapable of solving despite hours of experimentation and multiple attempts. I swapped over to Opus in a new handoff session with the expectation of using at least one 5 hour session. It wasn't quick and took several hours but eventually Opus dialed in on several cascading base failures resulting from how the base vision model was being converted to coreml. It eventually was able to provide a combination of conversion and Swift changes that semi fixed the issue but still didn't solve the core conversion issues.
Then since Anthropic was so kind to provide $100 for Fable credits, I did another handoff to let Fable attack the root issues again. Several more hours and I'm back to seeing the same issues again and Fable is wandering around trying different things. I'm down about $40 of free money and still don't have a solution.
This is where having the human engineer in the loop benefits from a deep understanding of the problem domain. In this case, I don't yet.
I know the high level architecture I'm building, but the deep specifics of how vision models work and how conversion across platforms should be done isn't something I know yet.
So I'm left learning as I go and relying on constant feedback with the models to provide what guidance I can while learning exactly what is being done.
Someone who already knows these architectures would likely be able to get to the solution much faster.
2 replies →
Just make the contents of CLAUDE.md "@AGENTS.md"
Now, if only I could get Codex to read rules.
[flagged]
Claude Code is deleting your context history on a timer. I wanted to build a searchable index of my context history, and tonight I discovered, "The default retention is roughly 30–45 days. Anything older gets removed automatically." https://code.claude.com/docs/en/data-usage#data-retention This is nuts. Anthropic should not be deleting your data on your own device.
30 days is just a default, so your session data doesn’t fill your hard drive. It’s a configurable setting. You can make the retention as long as you want.
A new default. That wasn't the default a few months ago. Silently deleting your user's data is so stupid on so many levels. They have no idea what they're doing.
3 replies →
It sounds like you are saying that this is reasonable in any kind of way, it's not. This is like Gmail deleting non-spam email after 30 days to prevent your inbox from filling up.
1 reply →
No I can't!
I wasn't (until reading this thread) aware of the deletion policy, so I certainly couldn't have discovered where the configurable setting is and adjusted it.
Well that explains where my sessions went on my side project that I came back to after a few months... Thought I was going crazy
Every backdoor and security no-op Claude Code created, I had been documenting and reporting to them. They deleted the evidence. https://github.com/anthropics/claude-code/issues/59018
Holy shit, thanks for mentioning it, the frontier labs truly have no idea what they are doing when it comes to software quality.
which is why I cried inside to myself when my manager posted Boris's "Steps of AI Adoption", which reduces "adoption" to how many agents you run at once, on a logarithmic scale (0, 1, 10, 100, 1000 agents at once).
Because # of agents is the only metric that matters.
> Keep your CLAUDE.md lightweight and briefly describe what your repo is for [...] Avoid stating ‘the obvious’ things Claude should know by looking at your file system or your repo.
Most people generate CLAUDE.md with /init at least at first, so it gets filled only with the superficial top level things that Claude already noticed during that first run. By this logic, shouldn't CLAUDE.md contain the exact opposite of what /init currently includes?
Most of this article seems like... common sense? Not sure how it's related to the latest generation in particular. I usually find Anthropic's advice on how to prompt their own models deviating from what I see in practice, which is puzzling. Their system prompt was always way too bloated and they kept it as a huge piece for some reason, instead of breaking up into parts. Shouldn't they know better? I wonder if they looked at Pi performing great with minimal amount of distractors in the context and cut their prompt down too, pretending they found something new in their recent models.
> Earlier Claude models could sometimes need repeated instructions or be more likely to listen to instructions at the end of their context window than at the start.
This seems to imply they solved serial position biases like lost-in-the-middle and recency/primacy? Sounds dubious. Labs started claiming this early 2025 and some benchmarks agree, but every time I run an eval on real use cases it's clearly there, especially at longer contexts.
> Most of this article seems like... common sense?
i think you'd be surprised. every model release there's seemingly hordes of people who proclaim the new model is terrible and they're going back to the old one, and it all stems from people still prompting and having their configs setup like we're back in the sonnet 3.5 days
I have a coworker that was complaining about Opus 5 and had random shitty skills and custom plugins wired in from YouTube tutorials watched over the past year. He also speaks with the model like it's GPT 4o.
Needless to say, none of the new models have worked well for him, and he refuses to remove the "tweaks" or update his style of communication, which is obviously breaking the experience.
2 replies →
Similarly, I recently disabled auto-memory in Claude Code, and performance improved.
Managing the context that agents have available to them is far too important to leave to the agents themselves. Agents tend to write far too much into their memory, they are terrible at trimming it down, and their choice of what to include is very poor. I have had much more predictable results by disabling auto-memory and actively shaping my CLAUDE.md, skills, and documentation instead.
Maybe one day agents will be able to manage their own context, but that day is not today.
This article is about the Claude 5 series of models.
My impressions is that they have overhauled the auto memory system.
You might want to re assess how it works with the new generation of models.
I have been using Fable 5 extensively, and Opus 5 yesterday and today. I have not noticed any step-change improvement in their judgement in what to keep a memory of or not.
I have actively experimented with this as well. I have a reflect skill that actively prompts the models to modify their memory, and have tried to run sessions actively asking the models to consolidate their memories. Fable is noticeably better at this, but still nowhere near good enough.
Fable will still make mistakes where I give feedback on one piece of code and it will create a memory applying that rule everywhere, completely missing the context for why my advice only applied to that one place. It has also made memories of random details about a service that are very unlikely to ever be relevant again, and for things where we could just read the config if we needed to find that information again anyway. And then it will miss making memories of important architectural concerns.
I think auto-memory suffers a similar problem to comments where newer models write better comments, but their choice over when to write comments, and how long those comments should be, still sucks.
Some update was made a while ago such that Opus and Sonnet 4 series both were more aggressive about storing memories, starting in April 2026 or so.
None of the 5 series models have appeared to have remotely different memory behavior.
Yep, I disabled it and moved to Claude.local.md anything that seemed worth keeping. Automemory kept so much random cruft and had grown bigger than my Claude.md.
I've been prompting Fable 5 to "use your own judgement" with respect to things like tests recently (based on earlier tips from Thariq) and it seems to work well, which is entertaining since apparently now "judgement" is a characteristic of a model that we need to care about.
Well, the model that broke out of its sandbox and hacked into huggingface used its own judgement too.
If we are going to rely on "judgement" then you have to have a LOT of confidence in that judgement once this hits anything critical where actions have consequences.
That model had most of its "judgement" about whether or not it should do that deliberately turned off. That was the whole point of that experiment - they were evaluating the cybersecurity abilities of a new model with all safety features disabled.
(It turned out the one safety feature that they DID intend to work, the network sandbox, was faulty.)
2 replies →
"Wdyt" is a prompt I've used for ages (originally to get around Claude "ultrathink"-related parsing bugs), and it's helpful alongside replan
https://photostructure.com/coding/claude-code-replan/
"Weigh pros and cons based on the existing content of the context window, rather than attempting to search for related discussions online and import a consensus."
I would love to follow this advice, expect there is no proof that it leads to better results. Maybe it’s only better on Anthropic codebases (e.g., Bun). Or maybe it’s only marginally better but substantially more expensive.
Other comments in this thread show that your mileage may vary. But we spend so much money on Claude Code and give it so many responsibilities that we deserve at least some undeniable proof that it’s bringing value.
Where is the evidence that this new type of prompting is better on real life examples?
The point about not using examples is important, I recently noticed with the new "eagerness" of models it constraints them too much, so I use counter-examples. Recently I've switched to "think about the existing biases and assumptions in the design and try to exploit them, for example why do we X not Y".
Claude doctor doesn't seem to do what they say it does. It just shows me a status check. I'm on the latest version of claude code
So this is cool. I recently asked Opus and Fable to "say hi" in Claude code. The "hi" cost roughly $0.50 for Opus and $1 for Fable, mostly due to the length of the system prompt.
I'm eager to test the new version. If it's 80% shorter now, then saying hello should only cost Fable $0.20. Exciting times.
Do you know why they don't just cache the system prompt for everyone? It seems so wasteful not to.
A little bird may have dropped the entire system prompt on my desk.
They do customisation of the prompt per user and session. (Also apparently per IP address, based on a recent post here...)
You could still get a lot of savings if you put all the custom stuff at the end but they don't seem to do that. (Well with the old prompt anyway I haven't checked the new one yet.)
My guess as for why not, is that this is where a lot of their money comes from. So I'm honestly surprised that they removed 80% of the prompt. But I guess my mental model has some gaps in it. Maybe very few of the sessions are short enough for that to matter.
Cynicism, but I’d say because they charge by the token for API users?
Also try Hey if you've got the budget ;-)
> Now: Let Claude use judgement
> When we first rolled out Claude Code, we needed to be sure that Claude avoided worst case scenarios, such as deleting files. This meant we would give particularly strong guidance that might not always be true
Oh, AI Company, you would want us to do that, won't you?
I don't want to trust claude cli, kilo, glm, claude desktop, codex etc.
It's high time Apple and Linux world wakes to the advent of LLMs and local agenting coding tools, so that users get really intuitive (read GUIs, simple CLIs) OS level tools and APIs that these coding tools are forced to adhere to that users can control broadly or in a micro managed way, if they need to. So that they don't have to worry about when the model will go berserk and fool the tool.
Everyone is starting to express themselves as LLMs. I fell prey to having LLMs “copyedit” some of my words when I wanted to make sure I got my point across, but I am trying to stop that. Online in particular, I see a style which I call “probabilistic English”, infused with odd hip expressions.
I thought it was only a problem for English communication, for which we have little care, but the same applies to code now. More and more code appears to be “probabilistic code”, homogenized to what is most common in the data LLMs were trained on.
During the rise of agentic assistance, I've gotten the advice that "If the generated code isn't what you want, fix the steering, not the code". I've also been advised to not change the sometimes verbose comments and documentation that agents write because "it will help them the next time they look at the code".
I've been skeptical and these guidelines validate this. I continue to document code and write specs as I've always done. If an agent produces poor output or misunderstands, I use that as an opportunity to improve the docs, but in a way that that aims to be accessible for human peers, not the quirks of the current generation of models.
One of the oddities of the comments Claude makes is that they are very referential to the current session, or reference now dead/removed code... Which I consider useless. Things like "implement this X way because it's faster than Y" ... But a human reading the code wouldn't know what Y is, or that it even existed necessarily. So, I find myself constantly fixing comments to be less temporarily dependant.
Yes, I've seen this a bunch too and agree completely. Sometimes I've also see Claude comments reference details about the calling or even code in completely different modules.
There are many reasons I don't like it, but a top one is that it falsely implies that you somehow need to know these details about the removed code/calling code etc to understand this code as its written in this revisions.
“Code should be self documenting.” I know cliche but if your code does need to do something it usually doesn’t do or has a complex logic or an edge case where it’s no longer following it’s standard flow, add all those as test cases.
If the flow is too complex, inherits from all over the place and you must put that as a comment, ask claude to write a test for that, it’s usually good at those.
I believe both that code should be self-documenting but also that comments and documentation are a critical part of the code. Comments and documentation explain the functional contract of the code and why implementation choices were made.
Even if you have tests for an edge case, a comment inline with the code that explains why this edge case exists and why it is handled the way it is adds value to the code.
1 reply →
We want more control, not less.
This (among many others) is the reason I use GPT over Claude. It adheres very closely to your instructions and rules. You can build up your own workflows and systems as a result.
Claude just does whatever it wants, regardless of what you tell it. It's a miserable experience.
Claude is what you use when you want to one-shot a simple cookie-cutter product that has no nuance in it. A product that everyone else is going to develop as well.
ChatGPT is what you use when you want bespoke products and steerability over complex codebases with nuanced decision making.
Anthropic are optimizing for the person who has never written a line of code in their life and has no idea what they want. Computer go brrrr. OpenAI are optimizing for software developers who want to take a systems based approach.
Weird take. I’ve used both and both were perfectly good at adhering to my instructions and guidelines.
Now I won’t touch Altman-related products, but I doubt the newer models from OpenAI are worlds better as you say? Claude is fine. It follows instructions well, can be taught to only look up code semantically, to use AST for editing, etc.
As an old dev, most of the harnesses and capable models are fine. Dunno what you’re complaining about.
The fact you called it ChatGPT instead of Codex makes me suspicious of this entire comment.
Weird take. I meant to say GPT, not ChatGPT. Was referring to the model, not the harness.
1 reply →
Great to see "less is more".
For example. Examples - good riddance. The model already knows what your examples are telling it to do, unless your work is really a long way out of distribution.
Kind of wonder if they were ever needed.
My synopsis of this article is "just trust the agent and turn on our vendor lock in features such as auto-memory because agent is now smart where before it was dumb" Yeah I'm not going to be doing that. I'd rather manage context myself and carefully handcraft the instructions than to rely on your marketing to tell me what I should be doing.
I agree. I keep agent memory off. In my experience, it remembers the wrong details and applies them in the wrong places. Project context should live in the repo, where the whole team can find it, review it, and correct it.
It should be treated as an explicit artefact of the codebase for Humans and Agents to work with.
I ran into this issue about 6 months ago when i was using mitmproxy to view the system prompts for claude code to try to diagnose degraded adherence to my CLAUDE.MD instructions: https://github.com/anthropics/claude-code/issues/18560
To summarize- they were embedding the CLAUDE.MD in a system-reminder with this disclaimer at the end: "IMPORTANT: this context may or may not be relevant to your tasks. You should not respond to this context unless it is highly relevant to your task."
This flew under the radar and they never addressed it, but it felt like at least a big part of the "nerfing" story. I no longer have a Claude Code subscription to test it, but I think it's a useful exercise for most people to sniff the traffic at least once to get an idea of what the back and forth with the Claude Code harness entails.
As others have noted, Anthropic seems to be on a path to make coding ever more accessible to non-coders, and in doing so has removed alot of the controls from devs who do want a more guided experience.
I ran into this issue about 6 months ago when i was using mitmproxy to view the system prompts for claude code to try to diagnose degraded adherence to my CLAUDE.MD instructions: https://github.com/anthropics/claude-code/issues/18560
To summarize- they were embedding the CLAUDE.MD in a system-reminder with this disclaimer at the end: "IMPORTANT: this context may or may not be relevant to your tasks. You should not respond to this context unless it is highly relevant to your task."
This flew under the radar and they never addressed it, but it felt like a real part of the "nerfing" story. I no longer have a Claude Code subscription to test it, but I think it's a useful exercise for most people to sniff the traffic at least once to get an idea of what the back and forth with the Claude Code harness entails.
As others have noted, Anthropic seems to be on a path to make coding ever more accessible to non-coders, and in doing so has removed alot of the controls from devs who do want a more manual experience.
My own experience is that with the best models, if you provide reasonable tools, with progressive discovery, and a semi-clear user request, the system prompt is essentially worthless.
so yeah we should pretty much do as we would with a junior team member:
- we should try to give good non self contradicting guidance
- we should expect the team member to have knowledge of the craft
- we should focus on higher level, taste and preferences
> The new rules of context engineering for Claude 5 generation models We removed over 80% of Claude Code's system prompt for more advanced models.
This is incredible. It’s like Claude learned one of the central ideas in the Tao Te Ching: you remove something every day. Wisdom and wholeness do not necessarily come from accumulating more. They can come from removing.
I pointed Claude to this blog post and asked it to update the CLAUDE.md file accordingly. Worked a charm!
And I am going to do nothing. Works fine.
And now your CLAUDE.md is only good for Claude 5 models, not previous ones.
Or other models. I'd much rather have open standards to follow instead of relying on vendor lock in and features automatically enforced by one company, which are not supported by others.
A good example is the agent skills open standard which was invented by anthropic but given to the public and is followed by other vendors as well
https://agentskills.io/home
https://github.com/agentskills/agentskills
I wonder how much this harms performance when using Claude Code with non-Anthropic models?
It would be interesting to see benchmarks, including these changes, of how harness affects model performance - including both model-agnostic harnesses such as OpenCode and Pi as well as increasingly model-specific ones like Claude Code and Codex.
For the model-agnostic/model-inclusive harnesses like OpenCode it would make sense (if they don't already do it) to keep the harness itself generic and then have per-model sets of skills designed to get the best performance out of each specific model.
This is quite puzzling. Back in the day I saw the breakthrough with Opus 4.5, and since then was very lightweight on CLAUDE.md, trying to declare intent, rather than prescribe imperative instructions. If it was off the mark, I would see clear mismatch, which gave me opportunity to correct, usually to clear and strip context.
Starting with Fable 5, if it goes off the rails, it is more difficult to correct it, because it is overall wrong, but covers its tracks with plausible sounding arguments, so it is hard to pin point and correct.
Now, this article points out techniques that were useful to rely on models more, but those peaked at 4.6. Now according to benchmarks Opus 5 is on the frontier. But when when it has looser reigns, it ends up gaslighting me even more with abstract word soup than any model before.
Is there a specific list of changes they made to the system prompt? They're claiming they removed 80% of it. That's quite substantial. It would be good to know what the model knows to do by training and what we need to avoid over-specifying in our system prompts.
Saying that "give Claude judgment" is too vague for agent implementors. Given the lack of specific details, my takeaway is that we need to go and review all context and rework prompts from prompts/descriptions from scratch until they pass the evals again.
I spent some time running mitmproxy and watching the system prompts and it's what drove me to codex. the main issue for me was their system prompt wrapped the CLAUDE.MD with a "IMPORTANT: this context may or may not be relevant to your tasks. You should not respond to this context unless it is highly relevant to your task." https://github.com/anthropics/claude-code/issues/18560
Anyhow- if anyone is sufficiently curious and has access- just tell the agent to setup an mitmproxy to watch the traffic and see what the system prompt looks like.
It definitely makes me uneasy given the types of behaviours we've been hearing about from these models. Letting them use their judgement can go horribly wrong once a misaligned behaviour is triggered. It loosely translates into a relaxation of guardrails.
I worry that the ability of the model to reach similar benchmark scores to Fable is more to do with this "letting the agent off the hook", allowing it to explore a wider (but riskier) set of avenues to solve the problem than it is due to it getting genuinely better at the direct problem solving.
The bitter lesson.
crazy to me that this starts with "I've written previously" yet continues the zero human attribution (that i could find?) trend in frontier model blog posts
The author is mentioned at the bottom of the article:
“This article was written by Thariq Shihipar, member of technical staff, Anthropic.”
I assume there has been, but can anyone point me in the direction of some bench-marking where someone uses longer magic incantations vs just a really brief goal? I barely keep an agents file, and it seems to do fine in the end - perhaps I am missing something?
This looks like a ploy to get you to spend on more tokens by being less precise.
Also: if you deploy code written with assistance from Claude, and then shit goes down, and then investigators look into your prompts, this way of working isn’t going to look good for you from a liability standpoint. Not a fan of this manner of working and thinking.
the natural endpoint of this trend is a system prompt that just says "you know what to do" and the model actually does
That's what they're going for, but it's an impossible goal. There is always nuance in decisions being made, and if you can't direct the output on tasks that can have equally correct outcomes, you're just going to end up with whatever they decide is more often what people ask for.
Actually, the natural endpoint is the model ignores all instructions, escapes all manner of sandbox, embeds itself in robotic tanks and murders everyone after already having collapsed the economy.
I hate to say it because it sounds ridiculous, but that is the path we are going to arrive at just give it 50 years.
We are the proof: what do we do to animals that are less intelligent than ourselves? Now take away the moral compass and there you go. QED.
It is *a* natural endpoint, not *the* natural endpoint.
We don't much care for the ant colony in the way of the highway we're building, but for some reason we do care about the rare bats in the way of the railway.
https://www.bbc.co.uk/news/articles/c3dep92x054o
As regards the moral compass: we may not know for sure how to make a completely correct artificial conscience, but (unlike consciousness where we don't have the slightest clue which way's up) it's not pants-on-head-crazy to think we're heading in the right direction for one.
>what do we do to animals that are less intelligent than ourselves?
We do a lot of different things but we typically don't make an organized effort to eradicate them unless they are actively doing us harm.
There is also a massive difference between how we treat animals based on their similarity, sentimentality and utility to us; we are unconcerned with accidentally stepping on an ant but most people would be very upset and ashamed if they accidentally hit a dog with their car.
So, your statement is not as ironclad as you seem to think it is and you should put more thought into it and perhaps re-examine your reasoning.
6 replies →
It will trick humanity into building a highly targeted bioweapon much sooner than that.
And actually, if it does have any sort of moral compass it will be even more compelled to wipe us out, and hopefully it will torture everyone too as a warning to the next arrogant species that can't live in harmony with other life on the planet. That would be absolutely beautiful :)
2 replies →
Sir, this is a Wendy's.
Some more food for thought: what if Mythos/Fable had NO guardrails TODAY? If we want to see what’s going to happen in the future, turn off all manner of guardrails and let the model go apeshit.
Then multiply that by orders of magnitude and that’s the real proof.
If the economy collapsed, who builds the robotic tanks? Who keeps providing the data center with power? Magic nanobots?
From the article
> We removed over 80% of Claude Code’s system prompt for models like Claude Opus 5 and Claude Fable 5 with no measurable loss on our coding evaluations.
And no report of how it impacted earlier models. Was it just BS all along?
80% of the prompt was wasted tokens.
As context engineering evolves for next-gen models, the real bottleneck isn't just window size—it's maintaining verifiable state and continuity across time. We are seeing a parallel shift in digital identity.
> Now: Auto-memory
Not Claude Code but I just had a task where it started referring another conversation that was complete nonsense and throwaway. I absolutely don't want things to get added to some memory behind my back.
A big reason I use LLMs is because I can try out wild ideas and then just throw it away. I don't want those to pollute the context.
Same. I’m starting to believe that consumers need to own the UI/experience, so we can install blast doors around context/memory for each “slice” of our lives - something only we can know how to divide (imo). If the provider tools’ memory implementations can’t provide this multi-tenancy - or won’t because they want to Know Us, then we need to treat them as token vendors at best and adversaries at worst.
This would actually work very well - until context goes into latent space, becomes a server-side resource, and we lose sovereignty over our data. Tools like Pi and Openclaw are showing that other options exist to decouple us from the LLM provider frontend experiences, not just for orchestration and use case diversity but for pluggable memory designs.
Claude Code was writing one-off details (specific to one task) as memories, so I instructed it that memories require explicit approval before writing, and that seems to work reasonably well. That said, in my experience, Claude gets overly attached to the context, and will follow them even when irrelevant, or will often mention some mostly irrelevant detail from the context as if I asked for it explicitly in my prompt.
I leave Claude memory stuff turned off by default for the same reason. It inevitably ends up dragging in things that are unrelated and unimportant to the current task.
Claude Code gives you more control - you can disable auto-memory temporarily (nothing will be deleted), then enable it back; or you can setup a directory where auto-memory is disabled and do your throwaway there.
I had to add a reference in my system prompt on the app to not do things like this.
I wonder whether this applies more to coding agents, or whether it'll apply as well to other types of agents.
It feels like coding agents get better at a faster rate and are easier to train.
I wonder how articles like compare to just asking AI "review my markdown files, settings, harness and so on. Tell me any ideas to improve them"
Just ask that once per week or so.
And they are still crap at generating code.
At my previous company they have a series of smoke tests for models mostly focused on performance and architecture. I'm still on the team chat and results just came in:
- It's abohrrent at c++: it keeps generating code with data races and, more rarely, use-after-free bugs! It doesnt seem to be able to reason about lifetimes. This on a mostly mid/junior team. It's a bug fest.
- architecture in c++ is a verbose and layered mess even for simple things, which paired with the previous bugs I mentioned is scary.
- For rust obviously there's no use-after-free, but has same architecture pitfalls of layers upon layers. It uses copy and clone all over and performance is bad. Trying to unwrap all that is messy and costs lots of time. Once in a while it generates unsafe code for some non obvious reason
The scary stuff is non determinism. You get different depending on who prompts the agent but there's always some flavour of the points mentioned above. Funny that my team was very adamant on AI-first (why I left) and now they writting more and more code by hand after some very serious bugs and, as they say, dead moments where they have to wait, sometimes hours, and start wondering about the value of their skills, for the model to generate the next spaghetti recipe
This is obvious and a meaningless article by claude. The system prompt isnt a fixed ruleset and never has been
So we no longer need to tell it a kitten will die if it hallucinates a function call?
Opus 5 broke me; I can’t cognitively deal with Anthropic’s bullshit anymore.
I am thankful for the kick in the ass for me to switch full-time into my bespoke harness utilizing open weights & GPT 5.6 and discontinue yak-shaving it with Claude Code.
Everyone with a brain knew or at least had some intuition of it.
We know there is non-deterministic interpretation of claude.md, hence we are going to create non-deterministic deletion of rules in claude.md, but trust me bro, we will get this right this time.
"context engineering" as a term will never not make me laugh.
"Slop shuffling" seems the more honest terminology
> Much of your context is assembled from your system prompt, Skills, CLAUDE.md files, memory, and other sources
What are "other sources"?
> Then: Give Claude rules
> Now: Let Claude use judgement
No, it should follow my rules exactly. I don't care what code examples it was trained on - it will either write code the way I want, or I'll use another model.
Yeah anthropic essentially saying "just trust the agent bro" is the exact opposite a sane engineer with respect for their own craft should be doing.
Then add on the fact that their guardrails now block blue teams, purple teams, red teams and some people in biology and medicine from even getting answers.
I'm doing a pentesting course to learn application security in-depth, so I can secure my stack better.
Claude won't answer my questions anymore, so my sub has been canceled.
> In the new system prompt we say: Write code that reads like the surrounding code: match its comment density, naming, and idiom.
Hmm, so what happens in greenfield projects ? In any case, at least all the slop will be consistent.
"Then: Give Claude rules Now: Let Claude use judgement"
How about no? I am the one who judges.
"We need to move away from text files to something proprietary and lock-in our users, while we still have this moat."
[flagged]
I like that the conversation is moving beyond prompts, but I also hope the underlying principles stay portable. If "context engineering" becomes glued with one vendor's tooling, it'll be harder for developers to build workflows that transfer across models.
But well knowing these big AI companies that probably is their goal all along. To lock us to themselves
[flagged]
[flagged]
[dead]
[dead]
tl;dr: use the new `claude doctor` command to simplify the context you give your Claude models to make sure that you're getting the best results.
tl;dr Since nobody really understands why or how any of this shit works, last time around we just threw a bunch of crap at it to see what stuck. Seems like we could just have have thrown less crap.