← Back to context

Comment by jeanlucas

7 days ago

I was wondering why my local tool to inspect coding agent sessions stopped working in some cases.

This is a really interesting engineering decision, I wonder how many people will want an encrypted external piece of instructions running on their machine.

28 comments

jeanlucas

Reply
EGreg  7 days ago

It seems their incentives aren’t exactly aligned with their users, including corporate users. Look at the latest statements from Alex Karp, and now Satya Nadella etc.

Is this user-hostile? Encrypting stuff from the user is what RIAA used to do with DRM, worried about copyright infringement.

  • pornel  7 days ago

    Nothing by Alex Karp is aligned with any users. This guy is a mercenary, and he seems to be in this job for the love of killing.

    • arjie  7 days ago

      Yeah but his users are hiring him to set up their killing machines so that they’ll do it better. So that makes him highly aligned with users.

      Besides, the SaaSification of these things is expected. When you run a model, the reasoning traces are an internal implementation detail of the program that then results in certain user-visible output. It can be used to distill etc and most users don’t care about it.

      It’s not some novel thing that internal implementation detail of software is a trade secret.

      1 reply →

  • echelon  7 days ago

    I think there are OpenAI / Anthropic employees in here flagging comments:

    https://news.ycombinator.com/item?id=48907099

    These companies are behaving against the best interests of their customers.

    Both OpenAI and Anthropic should be ashamed.

    These companies' products are awesome, but the way they conduct business is scummy.

    They must be worried there is no moat. If they keep up this behavior, there truly will be no moat. They're pushing people away.

    • EGreg  7 days ago

      It’s not about the companies specifically, or even AI specifically, but the incentive structure. I think the issue is more that they are embedded in a system of competition, where they can’t afford not to do things to “win” and fend off competitors’ practices. The people in the companies feel and understand this better than anyone.

      I think Anthropic, being a Public Benefit Corporation, can absolutely do better than OpenAI. OpenAI was a nonprofit but flipped. Many people left from OpenAI to Anthropic, I think all original founders of Anthropic are still with Anthropic.

      We have seen the same things in Web2 and Web3, this is not at all unique to AI companies! How many people at HN went to work for Facebook, or Google etc? The “don’t be evil” motto is nice but when you have to compete, you often end up doing the things you think you didn’t have to do.

      With Web2 the stakes were: “centrally controlling speech”. With Web3 the stakes were: “people lose programmable money they gambled with”. With AI the stakes are much bigger. This isn’t about a company being moral or not. This is about incentives of the broader ecosystem, and how the products are designed.

      As long as people (like Kevin O’Leary) say “but are you going to just let China pull ahead of us” we’re all going to be in a race (to the bottom, for a lot of affected people). It isn’t about OpenAI or Anthropic specifically. Hate the game, not the player.

      It doesn’t have to be this way. The game can change: https://safebots.ai/singularity.html

baby  7 days ago

if you're running in YOLO mode you already don't care, what matters are the tool calls and these can't be encrypted

  • Majromax  7 days ago

    No, you'd still care. YOLO mode is about instantaneous permissions and access control, inspection of subagent prompts is about retrospective quality control. If the main model is instructing subagents to do a subtly wrong thing, the overall process quality will degrade in ways that might be very hard to detect or fix without deep inspection of the middle stages.

  • embedding-shape  7 days ago

    I'm running in YOLO mode, not sure why that mean I don't want introspection into what the sub-agents are running? Everything is running in a constrained environment, so not sure why you'd need the harness to have limits.

fidotron  7 days ago

[flagged]