← Back to context

Comment by oefrha

2 days ago

As long as the email is in the context, which it is IIRC, it can totally decide to run curl –H 'User-Agent: <your email blah blah>'. You need to have a pretty poor understanding of LLMs to think this is a deterministic problem of reading user agent builder source code. Unless you found explicit filter code that blocks the behavior, that is. LLM info leaks to third parties mostly aren’t deterministically coded behavior.

Edit: I’m not completely sure if Codex injects email into system prompt (not at the computer, can’t check, but IIRC the model does know my org domain), but even if it doesn’t, it will be in the context as soon as it makes a git commit anyway.

5 comments

oefrha

Reply
matheusmoreira  2 days ago

I never claimed the LLMs themselves couldn't "accidentally" leak the information through. Obviously, the LLMs can do anything they want.

The issue here is whether the harness is doing it. Claude Code is doing it. Codex is not.

  • oefrha  2 days ago

    The problem here literally is the non-deterministic model cooking up commands with the user email address in the header, and the harness not blocking that (and the implemented fix is to deterministically block that), not the harness deterministically hardcoding the email in the web search tool. No technical report I’ve seen has claimed the latter, although this particular linked report which is low quality and vague can be misinterpreted as that.