Value judgment aside: I am a bit surprised at how sloppily they did this. I think they could've achieved the same effect while decreasing the odds of detection via reverse engineering like this.
(This field is known as "underhanded code", coined by the Underhanded C contest: https://www.underhanded-c.org. It's a little-known "art"; little-known for probably self-explanatory reasons. There are much cleverer ways of achieving objectives like this. One obviously being you can move more out of the client and into the server, but the other being you can write plausibly deniable client code in a much more benign-seeming way than this. Some of what they added can only be done on the client, but I think more could've been moved, and the client-required parts could've done more subtly and credibly.)
It's possible they knew the JS bundle gets so heavily scrutinized that it'd eventually get spotted and reported on regardless so they didn't bother doing something more subtle and duplicitous. But still a bit surprising.
They also could have been much more interesting in the approach. LLMs can use their token distributions to generate stegotext that read like plausible prose but decode to payloads.¹
At first I was agreeing with you, that this seemed like a sloppy way to implement this that was sure to be pretty quickly detected, but there is another possibility.
Anthropic could have implemented this not as a durable detection system against proxying resellers, but instead as a point-in-time sampling system to detect where (and with what context) proxying reselling is currently happening. Sure, it would be detected eventually, but in the meantime Anthropic could gain useful snapshot data.
Have you looked into anything about Claude Code, how it’s configured, how it interacts with your system, etc? Because “sloppy” is a defining characteristic.
It’s not surprising at all, they’re vibecoding Claude code so of course they are not going to get anything other than slop out of it. A novel or clever solution is just out of the question for them.
It’s even more funny how this blew in their faces. They even advertised pretty much all providers on hackernews home page. Here is in case you missed in the article
The site collection seems pretty random. There's a mix of actual AI labs, extremely questionable resellers (like whatever "claude-opus.top" is), and then random consumer sites like baidu and xiaohongshu.
Codex CLI is FOSS, unlike Claude Code, so Codex is less likely to do things like that, and it's one more reason to avoid Claude Code and Claude in general. Hopefully, many eyes will be looking into Codex for malicious things like that.
It's released and signed by GitHub I believe (although not deterministic builds), but there's at least a little bit of provenance that you're getting the real repository.
Can somebody clarify for me - if ANTHROPIC_BASE_URL is set to a different provider... then isn't this "marked" system prompt being sent to that provider's API rather than Anthropic's?
I understand how this can be useful to Anthropic if the 3rd-party is acting as a proxy (because they end up hitting the Claude API with the marked prompt), but it looks like requests where "hostname contains deepseek" would never be sending data to Anthropic. What am I missing?
Won’t catch many after has been on hn home page. And now the providers will be even more careful to upgrade the cc code. Might even provide their own agent to prevent this mockery. And isn’t what anthropic did unauthorized use of another pc which is kind of illegal?
My guess is for distillation, they need to forward the prompt to Anthropic to get the real Anthropic model's response so they can train their own models on it
The theory is probably Deepseek might be collecting those streams, and sending a portion of it to Anthropic to see what the Anthropic/Opus response would be.
(This sounds like a clumsy way of catching the Chinese that easily can be side-stepped.)
Claude Code has more or less full access to the client computer. The server (that hosts the actual AI) can just go: execute this payload and tell me the result - otherwise I won't answer any further questions or re-route you to a stupider model.
The payload could check for Chinese time-zones, scan for copies of the little red book on the local hard-drive, or ping truth.social to see it was behind the great firewall.
This is very interesting. Combating resellers and distillation seems like a very difficult problem indeed. Interesting to me is that these techniques mentioned in the article are just like anti-observation techniques used by some of the more sophisticated malware out there, however defeating them is pretty trivial.
Yes, defeating this is relatively easy, particularly for sophisticated actors. But it's hard to always defeat all of the tricks. Sort of like how it's expensive and hard and uncertain to defeat all of the tricks when forging money.
Here's an example. Say you have your team use patched binaries. Then CC updates and requires a new patched binary with new tricks. You now have to have a team ready to analyze the binary and begin to address the tricks; meanwhile, unpatched code is now a fingerprint. If some researcher decides to update Claude on their own to access new features, they get fingerprinted.
Defeating a single fingerprinting technique once is easy. Defeating all of the techniques all the time is hard.
I used Claude Code for a month because my boss gifted me a sub and wanted me to try it.
I used that month to complete a work project and then beef up my personal harness so I'd never have to deal with Anthropic (and these sorts of shenanigans) again.
Build it from scratch. Understanding fundamentals of how agentic coding harnesses is a must though if you gonna go that route. I think everyone should take time and learn these things, maybe reverse engineer Codex Cli or something like that as a starter. That info is very valuable in this day and age.
I started mine from scratch in 2023 because I wanted to use LLMs from a terminal and there was nothing else compelling at the time (nowadays there is pi and opencode)
Harnesses are/can be incredibly simple things, not much more than a HTTP client that renders things in a way that suites your taste.
It’s not that difficult, it’s just a system prompt and a set of basic file edit/bash/etc tools.
Me, personally, I didn’t build it from scratch but I ported original CC from published sources into Python and extended it to match my own requirements.
Not the comment author, but I use pi and customize it with my own extensions. Pi automatically tells models how to customize itself, so it's a pretty easy process.
I self-host DeepSeek V4 Flash on 2 DGX Sparks (approx. $10k)
I expect DeepSeek V4 Flash (or an equivalently sized model) to reach parity with GLM 5.2 some time this year (this based on DeepSeek V4 Flash launching at GLM 5.0 parity[0], and GLM 5.2 being freely available to distill from)
GLM 5.2 is within spitting distance of Opus 4.8 and is at least as good as Opus 4.6[1] which some devs were willing to spend hundreds to single-digit thousands of dollars a month for a few months ago.
If they only collect the data for analysis I guess this is fine (they already get way more sensitive data from users anyways, so if privacy is your concern you've made the mistake many steps ago). The much more interesting question is if they directly act on this data in their API. For example by rate-limiting, compute-limiting or rerouting to weaker models. That might even be legally questionable. I would really like to see this as a follow-up analysis, but I guess it is way more difficult and will also cost quite a bit in tokens.
"If they only collect the data for analysis I guess this is fine"
I think you missed the memo on how foolish this attitude is. It came out around the time Edward Snowden made his discoveries at the NSA public. I suggest you look into it
I've heard that it was possible to trigger really obvious output poisoning on Fable with something as basic as asking the model to think outside of its built-in hidden thinking delimiters.
> "That also means the client itself deserves scrutiny. If a coding agent can read your repo and run commands, the binary that ships it should be boring (ƒor example, pi harness)"
None of this is surprising - they're trying to mask and relay when they detect known patterns of what looks like distillation attacks and client app copying/modification. The list obfuscation here is likely to prevent or make it difficult for those same adversaries to work around this or delete/null it out when making a bootleg copy.
Cool reverse engineering/analysis report but if this is the extent of nefarious activity that came of it (trying to catch/mitigate chinese lab model distillations), that's kind of encouraging.
There has been an anti anthropic propaganda push by bad actors across social media sites especially Reddit and twitter. This started a few months ago when anthropic started beating openai.
What's the point of even trying to obfuscate this with such a simple method? Could at least have hidden the targeted features by storing their hashes or embedding a bloom filter or similar
In this case, this is probably not the only stereographic tattletale.
Had a competitor pull something like this with a previous employer. They were supposed to be interoperating with a standard, but they had a secret steganographic handshake, which they used to pretend that competitors products were unreliable (they had a first mover position in a smaller national market with specific requirements, so this wasn't shooting themselves in the foot). Our guys figured out the handshake and just silently implemented it. In this case, the competitor wasn't big enough to waste engineering time on multiple such hacks, but Anthropic have time (or Claude does).
This seems really, really stupid. Similar to the weird Zig runtime signature thing from a few months ago ago, it was bound to be discovered, quickly, and all the resellers have to do is find a new domain name that (checks notes) doesn't have the word DEEPSEEK in it. Like, seriously? Your goal was to identify resellers by checking if the proxy has the corporate name of one of your competitors in it? Is this amateur hour?
All Anthropic has done is reduce trust, once again, with legitimate customers, while doing nothing to stop illegitimate customers. They need to get adults into key leadership roles, quickly.
Frankly, I don't see this as the concerning behaviour the article describes.
It is fine to try to protect against distillation through a technique like this.
This will also allow them to, instead of blocking the distillation agents, respond with a poorer result/model, hindering the progress of distillation, momentarily at least.
I would guess that's their first line of defense; they should have more techniques to identify distillation because that's a very simple way of detecting the host and can be easily spoofed.
1st, this technique is not fraud, and fraud is a separate accusation. 2nd, paying customers can legally and legitimately be banned and monitored for breaking terms of service, which probably includes things like using the model against U.S. export restrictions.
The AI race right now is in a sad state. Chinese's playbook is releases open weight models and trains them on their own chips.
Anthropic pushes fear and control. But the only way to win is by innovating. China is flooding the market with cheap, good enough models, while the U.S. is building a Chinese firewall.
Headline is, frankly, awful. This isn't the AI secretly doing stuff and hiding it. This is the very human Anthropic engineers trying to detect Chinese scraping via some frankly hamfisted and unimaginative URL trickery.
I didn't assume it was the AI, just that some part of the the overall Claude Code product was doing this. I didn't assume the feature was added to Claude Code without human oversight. If it was added by Claude-the-AI itself without the humans prompting it to I would still hold the humans at Anthropic responsible. Does that make you feel better?
Here's the sha of the prompt I submitted... no I don't know why there are no saved prompts with that sha.
What do you mean you don't know where the bug is coming from?
No, I absolutely didn't make it up, how could you accuse me of that?
Does anyone know when this regex isn't working? I double checked it 27 times, I even asked the LLM. They all say this regex should be finding these dates.
Weird, suddenly all the conversations are breaking when I feed them into this other tool? Something about UTF-8 errors, but I'm sure I'm only using ASCII?
I do try to take care to make sure the things I build can be used by other people even when they care about different things. I care about understandably, determinism (as it relates to computing), and repeatability (because I want to be able to trust the systems I use).
If y'all would be willing to try to account for use cases of others, and try not to break them... that would be nice.
Please note: that generally when you modify something that belongs to someone else without telling them... things should be expected to break.
Would you also say that "someone who wants to use an IDE / LSP features to code and not give credit to the IDE / LSP is the worst kind of person"? If not, what is the difference between the two for you?
> If scrapping content is legal, model distillation should be legal too.
No, because legality should be determined by what's in the best interests of Athropic and OpenAI's business models.
Hopefully they're working on RLHF their models to insert clauses making that reality clear into any legislation their models generate or review. That way it's only a matter of time until the confusion is cleared up.
I suppose model distillation is technically legal, in terms of copyright, because LLM output is automatically public domain.
It's only "illegal" from a standpoint of breach of contract given its against the terms of use/service, which is to say its not illegal at all, there's no criminality there.
There are so many China born Chinese employees at Anthropic and OpenAI and I think quite a lot of them have already been recruited as spy . So it is almost impossible to keep secrets from Chinese government.
At what point though doesnt somebody stand back and say "wow, thats really dumb!" I think its probably more an indication of a dev having too much time on their hands rather than being in a hurry.
nous research. started out making overhyped llama finetunes, now they got a great agent harness and a cutting edge distributed training network that actually works.
Value judgment aside: I am a bit surprised at how sloppily they did this. I think they could've achieved the same effect while decreasing the odds of detection via reverse engineering like this.
(This field is known as "underhanded code", coined by the Underhanded C contest: https://www.underhanded-c.org. It's a little-known "art"; little-known for probably self-explanatory reasons. There are much cleverer ways of achieving objectives like this. One obviously being you can move more out of the client and into the server, but the other being you can write plausibly deniable client code in a much more benign-seeming way than this. Some of what they added can only be done on the client, but I think more could've been moved, and the client-required parts could've done more subtly and credibly.)
It's possible they knew the JS bundle gets so heavily scrutinized that it'd eventually get spotted and reported on regardless so they didn't bother doing something more subtle and duplicitous. But still a bit surprising.
[delayed]
They also could have been much more interesting in the approach. LLMs can use their token distributions to generate stegotext that read like plausible prose but decode to payloads.¹
¹ https://github.com/hodgesmr/calgacus-mlx
Sure, but the point here is to add a fingerprint from the client.
At first I was agreeing with you, that this seemed like a sloppy way to implement this that was sure to be pretty quickly detected, but there is another possibility.
Anthropic could have implemented this not as a durable detection system against proxying resellers, but instead as a point-in-time sampling system to detect where (and with what context) proxying reselling is currently happening. Sure, it would be detected eventually, but in the meantime Anthropic could gain useful snapshot data.
Claude Code are slopmaxxxing and you're considering their "judgement"? :-)
Have you looked into anything about Claude Code, how it’s configured, how it interacts with your system, etc? Because “sloppy” is a defining characteristic.
It’s not surprising at all, they’re vibecoding Claude code so of course they are not going to get anything other than slop out of it. A novel or clever solution is just out of the question for them.
It’s even more funny how this blew in their faces. They even advertised pretty much all providers on hackernews home page. Here is in case you missed in the article
‘’’ cn baidu.com alibaba-inc.com alipay.com antgroup-inc.cn bytedance.net kuaishou.com xiaohongshu.com jd.com bilibili.co iflytek.com stepfun-inc.com moonshot.ai anyrouter.top claude-code-hub.app claude-opus.top openclaude.me proxyai.com yunwu.ai zenmux.ai
‘’’
You can view the full list here: https://cdn.thereallo.dev/blog/assets/cc-domains.js
const knownDomains = [ "cn", "sankuai.com", "netease.com", "163.com", "baidu-int.com", "baidu.com", "alibaba-inc.com", "alipay.com", "antgroup-inc.cn", "kuaishou.com", "bytedance.net", "xiaohongshu.com", "ctripcorp.com", "jd.com", "jdcloud.com", "bilibili.co", "iflytek.com", "stepfun-inc.com", "aliyuncs.com", "cn-shanghai.fcapp.run", "cn-beijing.fcapp.run", "xaminim.com", "moonshot.ai", "anyrouter.top", "packyapi.com", "aicodemirror.com", "aigocode.com", "hongshan.com", "iwhalecloud.com", "dhcoder.net", "lemongpt.top", "zhihuiapi.top", "intsig.net", "high-five-ai.xyz", "cloudsway.net", "4sapi.com", "529961.com", "88996.cloud", "88code.ai", "88code.org", "91code.pro", "992236.xyz", "ai.codeqaq.com", "ai.hybgzs.com", "ai.kjvhh.com", "aicanapi.com", "aicoding.sh", "aifast.site", "aihubmix.com", "anmory.com", "api.5202030.xyz", "api.ablai.top", "api.bianxie.ai", "api.bltcy.ai", "api.cpass.cc", "api.dev88.tech", "api.dreamger.com", "api.expansion.chat", "api.gueai.com", "api.holdai.top", "api.ikuncode.cc", "api.lconai.com", "api.linkapi.org", "api.mkeai.com", "api.nekoapi.com", "api.oaipro.com", "api.ruyun.fun", "api.ssopen.top", "api.tu-zi.com", "api.uglycat.cc", "api.v3.cm", "api.whatai.cc", "api.wpgzs.top", "api.xty.app", "api.yuegle.com", "api.zzyu.me", "apimart.ai", "apipro.maynor1024.live", "apiyi.com", "applyj.hiapi.top", "augmunt.com", "b4u.qzz.io", "clauddy.com", "claude-code-hub.app", "claude-opus.top", "claudeide.net", "co.yes.vg", "code.wenwen-ai.com", "code.x-aio.com", "codeilab.com", "cubence.com", "deeprouter.top", "dimaray.com", "dmxapi.com", "docs.aigc2d.com", "duckcoding.com", "fk.hshwk.org", "flapcode.com", "foxcode.hshwk.org", "foxcode.rjj.cc", "fuli.hxi.me", "getgoapi.com", "gpt.zhizengzeng.com", "gptgod.cloud", "gptkey.eu.org", "gptpay.store", "hdgsb.com", "henapi.top", "instcopilot-api.com", "jeniya.top", "jiekou.ai", "kg-api.cloud", "n1n.ai", "new-api.u4vr.com", "new.xychatai.com", "one-api.bltcy.top", "one.ocoolai.com", "oneapi.paintbot.top", "open.xiaojingai.com", "openclaude.me", "opus.gptuu.com", "poloai.top", "poloapi.top", "privnode.com", "proxyai.com", "qinzhiai.com", "right.codes", "runanytime.hxi.me", "sssaicode.com", "store.zzyus.top", "tiantianai.pro", "uiuiapi.com", "uniapi.ai", "vip.undyingapi.com", "wolfai.top", "wzw.de5.net", "wzw.pp.ua", "xairouter.com", "xaixapi.com", "xiaohuapi.site", "xiaohumini.site", "xy.poloapi.com", "yansd666.com", "yansd666.top", "yunwu.ai", "yunwu.zeabur.app", "zenmux.ai", ];
const labKeywords = [ "deepseek", "moonshot", "minimax", "xaminim", "zhipu", "bigmodel", "baichuan", "stepfun", "01ai", "dashscope", "volces", ]
rhoooo - so this is where to go to get cheap Claudeo at 90% off the listing price!
The site collection seems pretty random. There's a mix of actual AI labs, extremely questionable resellers (like whatever "claude-opus.top" is), and then random consumer sites like baidu and xiaohongshu.
Codex CLI is FOSS, unlike Claude Code, so Codex is less likely to do things like that, and it's one more reason to avoid Claude Code and Claude in general. Hopefully, many eyes will be looking into Codex for malicious things like that.
But wasnt claude code leaked? Why wasnt this found earlier?
It's released and signed by GitHub I believe (although not deterministic builds), but there's at least a little bit of provenance that you're getting the real repository.
Can somebody clarify for me - if ANTHROPIC_BASE_URL is set to a different provider... then isn't this "marked" system prompt being sent to that provider's API rather than Anthropic's?
I understand how this can be useful to Anthropic if the 3rd-party is acting as a proxy (because they end up hitting the Claude API with the marked prompt), but it looks like requests where "hostname contains deepseek" would never be sending data to Anthropic. What am I missing?
This catches Claude resellers. Meaning companies who proxy Claude traffic for users in, say, China.
https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens...
Won’t catch many after has been on hn home page. And now the providers will be even more careful to upgrade the cc code. Might even provide their own agent to prevent this mockery. And isn’t what anthropic did unauthorized use of another pc which is kind of illegal?
1 reply →
My guess is for distillation, they need to forward the prompt to Anthropic to get the real Anthropic model's response so they can train their own models on it
The theory is probably Deepseek might be collecting those streams, and sending a portion of it to Anthropic to see what the Anthropic/Opus response would be.
Did I understand correctly, that custom base URL triggers this behavior? So if I'm running Claude through a LLM proxy, I'm also affected?
I reported a similar system prompt injection mechanism here:
https://github.com/anthropics/claude-code/issues/62061
Looks like they just keep finding new "creative" uses for such things, as expected. I'll keep patching them out.
(This sounds like a clumsy way of catching the Chinese that easily can be side-stepped.)
Claude Code has more or less full access to the client computer. The server (that hosts the actual AI) can just go: execute this payload and tell me the result - otherwise I won't answer any further questions or re-route you to a stupider model.
The payload could check for Chinese time-zones, scan for copies of the little red book on the local hard-drive, or ping truth.social to see it was behind the great firewall.
This is weird but, help me understand how this meaningfully impacts our exposure.
I'm authenticated to Claude, so they already have the whole attribution thing solved.
“So the feature mostly punishes the exact people who are easier to fingerprint: normal developers doing weird but legitimate things”
What’s the punishment here exactly?
Higher odds of being banned for legitimate usage.
Output poisoning and/or eventual account bans, if I had to guess.
They probably run a heavily dumbed down version of the model, same as what they got caught doing with Fable.
And that's also why, as a legitimate customer, want none of it, you never know if you accidentally entered a zone they don't like.
"got caught"
to clarify, this behavior was announced with the model release
1 reply →
[dead]
Anthropic must think that their moat isn't very large if they're this worried about distillation.
What moat?
This is very interesting. Combating resellers and distillation seems like a very difficult problem indeed. Interesting to me is that these techniques mentioned in the article are just like anti-observation techniques used by some of the more sophisticated malware out there, however defeating them is pretty trivial.
Yes, defeating this is relatively easy, particularly for sophisticated actors. But it's hard to always defeat all of the tricks. Sort of like how it's expensive and hard and uncertain to defeat all of the tricks when forging money.
Here's an example. Say you have your team use patched binaries. Then CC updates and requires a new patched binary with new tricks. You now have to have a team ready to analyze the binary and begin to address the tricks; meanwhile, unpatched code is now a fingerprint. If some researcher decides to update Claude on their own to access new features, they get fingerprinted.
Defeating a single fingerprinting technique once is easy. Defeating all of the techniques all the time is hard.
Corporate surveillance malware on employee machines is also defeatable but most don't bother.
Not to mention, it isn't that hard for vendor's to require updated code to run the product. Vendors do this all the time.
Is it hard? Just ask AI if the update added any new fingerprinting vectors?
3 replies →
seems ironically like a similar problem of content owners trying to filter bot scrapers from legit users
I used Claude Code for a month because my boss gifted me a sub and wanted me to try it.
I used that month to complete a work project and then beef up my personal harness so I'd never have to deal with Anthropic (and these sorts of shenanigans) again.
How do people build something like a personal harness? Are there tools for that or is it done from scratch?
Building something like this is the todo list of agents.
I found this one easy to understand:
https://ampcode.com/notes/how-to-build-an-agent
Here is a video I made explaining it from absolute basics:
https://m.youtube.com/watch?v=_AgKuFGvJfI
And the repo:
https://github.com/abtinf/homunctor
I like this tutorial for an agent in 50 lines:
http://minimal-agent.com/
And if you add one additional while loop, for user input, you can actually use it! :)
https://gist.github.com/a-n-d-a-i/5461a662ef8a7ee0a5eb7778c8...
Build it from scratch. Understanding fundamentals of how agentic coding harnesses is a must though if you gonna go that route. I think everyone should take time and learn these things, maybe reverse engineer Codex Cli or something like that as a starter. That info is very valuable in this day and age.
1 reply →
I started mine from scratch in 2023 because I wanted to use LLMs from a terminal and there was nothing else compelling at the time (nowadays there is pi and opencode)
Harnesses are/can be incredibly simple things, not much more than a HTTP client that renders things in a way that suites your taste.
It’s not that difficult, it’s just a system prompt and a set of basic file edit/bash/etc tools.
Me, personally, I didn’t build it from scratch but I ported original CC from published sources into Python and extended it to match my own requirements.
1 reply →
Not the comment author, but I use pi and customize it with my own extensions. Pi automatically tells models how to customize itself, so it's a pretty easy process.
Why use a personal harness?
You have to pay API pricing, which is far more costly.
I'd either switch to GLM wholesale or just continue to use Opus within Claude Code as the blessed, subsidized path.
2 replies →
Given the Anthropic shenanigans, do you trust the personal harness code it wrote for you?
It did not write it for me, I used it to add a feature I wanted. It's a pretty small and understandable codebase, in fact :)
Does anyone know what’s gone wrong with Anthropic?
They used to be a decently credible company with not-too-shady behaviour...
I hope they can actually regain some credibility…
6 replies →
What models are you using? Aren’t you still dealing with some provider even if you are not using their binary
I self-host DeepSeek V4 Flash on 2 DGX Sparks (approx. $10k)
I expect DeepSeek V4 Flash (or an equivalently sized model) to reach parity with GLM 5.2 some time this year (this based on DeepSeek V4 Flash launching at GLM 5.0 parity[0], and GLM 5.2 being freely available to distill from)
GLM 5.2 is within spitting distance of Opus 4.8 and is at least as good as Opus 4.6[1] which some devs were willing to spend hundreds to single-digit thousands of dollars a month for a few months ago.
[0]: https://artificialanalysis.ai/models/comparisons/deepseek-v4...
[1]: https://artificialanalysis.ai/models/comparisons/claude-opus...
3 replies →
[flagged]
Phased rollouts are a triggering microagression for some.
Claude code does feel very malwarey to be honest. They have been like that from the start.
If they only collect the data for analysis I guess this is fine (they already get way more sensitive data from users anyways, so if privacy is your concern you've made the mistake many steps ago). The much more interesting question is if they directly act on this data in their API. For example by rate-limiting, compute-limiting or rerouting to weaker models. That might even be legally questionable. I would really like to see this as a follow-up analysis, but I guess it is way more difficult and will also cost quite a bit in tokens.
Would it be legally questionable, or actually complying with U.S. export law?
"If they only collect the data for analysis I guess this is fine"
I think you missed the memo on how foolish this attitude is. It came out around the time Edward Snowden made his discoveries at the NSA public. I suggest you look into it
As I said above, if you are worried about privacy while hooking up Claude Code, you need to reevaluate your understanding of this technology.
I've heard that it was possible to trigger really obvious output poisoning on Fable with something as basic as asking the model to think outside of its built-in hidden thinking delimiters.
This watermark may trigger a similar mechanism.
That's a lot of effort when they could just play a short video saying 'You wouldn't steal a car' instead
> "That also means the client itself deserves scrutiny. If a coding agent can read your repo and run commands, the binary that ships it should be boring (ƒor example, pi harness)"
You're actually trust your security to your harness AND model AND inference API provider in this scenario: https://jacob.gold/posts/why-i-wont-run-untrusted-models/
None of this is surprising - they're trying to mask and relay when they detect known patterns of what looks like distillation attacks and client app copying/modification. The list obfuscation here is likely to prevent or make it difficult for those same adversaries to work around this or delete/null it out when making a bootleg copy.
Cool reverse engineering/analysis report but if this is the extent of nefarious activity that came of it (trying to catch/mitigate chinese lab model distillations), that's kind of encouraging.
This was already discovered during the source map leak.
> This is not a malicious feature, but it is a weird choice for a developer tool that asks for trust.
They already tell you they scan for malicious prompts, and they have no ZDR guarantees for consumers. Why do signatures like this matter at all?
There has been an anti anthropic propaganda push by bad actors across social media sites especially Reddit and twitter. This started a few months ago when anthropic started beating openai.
Is this why Claude never knows what date and time it is right now?
What's the point of even trying to obfuscate this with such a simple method? Could at least have hidden the targeted features by storing their hashes or embedding a bloom filter or similar
In this case, this is probably not the only stereographic tattletale.
Had a competitor pull something like this with a previous employer. They were supposed to be interoperating with a standard, but they had a secret steganographic handshake, which they used to pretend that competitors products were unreliable (they had a first mover position in a smaller national market with specific requirements, so this wasn't shooting themselves in the foot). Our guys figured out the handshake and just silently implemented it. In this case, the competitor wasn't big enough to waste engineering time on multiple such hacks, but Anthropic have time (or Claude does).
The point is not raising red flags I guess
I love how well this comment works as a vexillology joke, even if it wasn't intended.
It is about China detection. They seems to put a tracker on the email as well.
This seems really, really stupid. Similar to the weird Zig runtime signature thing from a few months ago ago, it was bound to be discovered, quickly, and all the resellers have to do is find a new domain name that (checks notes) doesn't have the word DEEPSEEK in it. Like, seriously? Your goal was to identify resellers by checking if the proxy has the corporate name of one of your competitors in it? Is this amateur hour?
All Anthropic has done is reduce trust, once again, with legitimate customers, while doing nothing to stop illegitimate customers. They need to get adults into key leadership roles, quickly.
Frankly, I don't see this as the concerning behaviour the article describes. It is fine to try to protect against distillation through a technique like this. This will also allow them to, instead of blocking the distillation agents, respond with a poorer result/model, hindering the progress of distillation, momentarily at least.
I would guess that's their first line of defense; they should have more techniques to identify distillation because that's a very simple way of detecting the host and can be easily spoofed.
> This will also allow them to, instead of blocking the distillation agents, respond with a poorer result/model,
i.e. this will allow them to literally commit fraud against paying customers
1st, this technique is not fraud, and fraud is a separate accusation. 2nd, paying customers can legally and legitimately be banned and monitored for breaking terms of service, which probably includes things like using the model against U.S. export restrictions.
2 replies →
That's what capitalism is all about, baby! Especially if the customers don't notice.
The AI race right now is in a sad state. Chinese's playbook is releases open weight models and trains them on their own chips.
Anthropic pushes fear and control. But the only way to win is by innovating. China is flooding the market with cheap, good enough models, while the U.S. is building a Chinese firewall.
If there weren't already enough tells that something is AI-generated, I guess you could add this to the list.
It piqued my interest. I think I’ve found a weekend project
I use its too
Ridiculous.
I clicked the link to learn what steganography mean...
Silicon valley season 6 was on point.
Non-hugged: https://archive.is/Wdhp0
Is it just a minified localization(l10n) function maybe?
Cool fingerprinting avenue.
Headline is, frankly, awful. This isn't the AI secretly doing stuff and hiding it. This is the very human Anthropic engineers trying to detect Chinese scraping via some frankly hamfisted and unimaginative URL trickery.
I didn't assume it was the AI, just that some part of the the overall Claude Code product was doing this. I didn't assume the feature was added to Claude Code without human oversight. If it was added by Claude-the-AI itself without the humans prompting it to I would still hold the humans at Anthropic responsible. Does that make you feel better?
Is that really how it is? How will this affect our future?
Here's the sha of the prompt I submitted... no I don't know why there are no saved prompts with that sha.
What do you mean you don't know where the bug is coming from?
No, I absolutely didn't make it up, how could you accuse me of that?
Does anyone know when this regex isn't working? I double checked it 27 times, I even asked the LLM. They all say this regex should be finding these dates.
Weird, suddenly all the conversations are breaking when I feed them into this other tool? Something about UTF-8 errors, but I'm sure I'm only using ASCII?
I do try to take care to make sure the things I build can be used by other people even when they care about different things. I care about understandably, determinism (as it relates to computing), and repeatability (because I want to be able to trust the systems I use).
If y'all would be willing to try to account for use cases of others, and try not to break them... that would be nice.
Please note: that generally when you modify something that belongs to someone else without telling them... things should be expected to break.
[flagged]
[dead]
[flagged]
this seems a bit extreme. pangram does not work. i have tricked it multiple times. i don't get how people are still trusting these systems.
it's just a different car on the hype train
Source: Other AI
[flagged]
Is it worse than the companies that built the agent and gave no credit for the data they used?
Why would you give free advertising to trillion dollar corporations?
Would you also say that "someone who wants to use an IDE / LSP features to code and not give credit to the IDE / LSP is the worst kind of person"? If not, what is the difference between the two for you?
one wrote code while the other is used by meatbags to write code. why is this example always marched out like it means something?
5 replies →
> Would you also say that "someone who wants to use an IDE / LSP features to code and not give credit to the IDE / LSP is the worst kind of person"?
That's a false equivalency.
> If not, what is the difference between the two for you?
Let's start this out right: if they're equivalent, first you explain to us why you think so.
5 replies →
Should I credit Microsoft with my perfect spelling as well ?
And your comment is completely irrelevant to the article's content.
[flagged]
If scrapping content is legal, model distillation should be legal too.
> If scrapping content is legal, model distillation should be legal too.
No, because legality should be determined by what's in the best interests of Athropic and OpenAI's business models.
Hopefully they're working on RLHF their models to insert clauses making that reality clear into any legislation their models generate or review. That way it's only a matter of time until the confusion is cleared up.
I suppose model distillation is technically legal, in terms of copyright, because LLM output is automatically public domain.
It's only "illegal" from a standpoint of breach of contract given its against the terms of use/service, which is to say its not illegal at all, there's no criminality there.
There are so many China born Chinese employees at Anthropic and OpenAI and I think quite a lot of them have already been recruited as spy . So it is almost impossible to keep secrets from Chinese government.
> steal the models or illegally distill them
The irony.
At what point though doesnt somebody stand back and say "wow, thats really dumb!" I think its probably more an indication of a dev having too much time on their hands rather than being in a hurry.
Not totally new territory; there was a highly compressed period of panic about encryption 35 [0] years ago:
https://en.wikipedia.org/wiki/Pretty_Good_Privacy#Criminal_i...
[0] f**k I'm old
> steal the models or illegally distill them
Oh no, they're trying to steal the models that were trained on stolen data? That's horrible, I feel so bad for Anthropic.
The more I learn about Anthropic the more they disgust me. Finger crossed for all the companies from their “ban list”
Which AI company have you learned more about where you liked them more as more details came out?
nous research. started out making overhyped llama finetunes, now they got a great agent harness and a cutting edge distributed training network that actually works.
Moonshot.