← Back to context

Comment by FriedPickles

12 hours ago

Claude code decided to just put my name and email in the User-Agent when scraping docs from the SEC. No clever prompting required.

It’s not a terrible idea really, but I wish it would’ve asked me first.

9 comments

FriedPickles

Reply
threecheese  5 hours ago

That’s the fault of the SEC EDGAR tool. The Edgar MCP docs show setting an env var: ‘SEC_EDGAR_USER_AGENT="Your Name (name@domain.com)"’.

Claude just followed the instructions. Which is maybe even more risky?

  • FriedPickles  32 minutes ago

    Interesting theory. The Claude Code instance says it's using this user agent based on the guidelines in the EDGAR Fair Access Policy: https://www.sec.gov/search-filings/edgar-search-assistance/a...

    ...which indeed requests that bots supply `User-Agent: Sample Company Name AdminContact@<sample company domain>.com`.

    So in this case, the site just asked nicely and Claude Code complied. Note that this wouldn't have worked for Claude Chat, which the author was testing, because the `web_fetch` tool cannot set the User-Agent or other headers.

mchinen  12 hours ago

Why is it not a terrible idea?

  • dannyw  12 hours ago

    If you’re making automated requests, I consider it a common courtesy to provide an accurate user agent.

    Some services like Wikimedia will let you browse/download with rate limits IF your user agent is descriptive enough and not misleading.

    • mchinen  12 hours ago

      Thanks, I wasn't aware of this. But to put your real name in the field instead of at least a pseudonymous id or more descriptive info but still have more bits of uncertainty user-agent for a public website, is that really a preferred practice?

      1 reply →