← Back to context

Comment by EGreg

7 days ago

It seems their incentives aren’t exactly aligned with their users, including corporate users. Look at the latest statements from Alex Karp, and now Satya Nadella etc.

Is this user-hostile? Encrypting stuff from the user is what RIAA used to do with DRM, worried about copyright infringement.

10 comments

EGreg

Reply
pornel  7 days ago

Nothing by Alex Karp is aligned with any users. This guy is a mercenary, and he seems to be in this job for the love of killing.

  • arjie  7 days ago

    Yeah but his users are hiring him to set up their killing machines so that they’ll do it better. So that makes him highly aligned with users.

    Besides, the SaaSification of these things is expected. When you run a model, the reasoning traces are an internal implementation detail of the program that then results in certain user-visible output. It can be used to distill etc and most users don’t care about it.

    It’s not some novel thing that internal implementation detail of software is a trade secret.

    • Espressosaurus  7 days ago

      Except I care about the reasoning trace because I can stop a rathole that burns $50 of tokens as it chews for 10 minutes before I get the desired feedback that it’s done the wrong thing.

echelon  7 days ago

I think there are OpenAI / Anthropic employees in here flagging comments:

https://news.ycombinator.com/item?id=48907099

These companies are behaving against the best interests of their customers.

Both OpenAI and Anthropic should be ashamed.

These companies' products are awesome, but the way they conduct business is scummy.

They must be worried there is no moat. If they keep up this behavior, there truly will be no moat. They're pushing people away.

  • EGreg  7 days ago

    It’s not about the companies specifically, or even AI specifically, but the incentive structure. I think the issue is more that they are embedded in a system of competition, where they can’t afford not to do things to “win” and fend off competitors’ practices. The people in the companies feel and understand this better than anyone.

    I think Anthropic, being a Public Benefit Corporation, can absolutely do better than OpenAI. OpenAI was a nonprofit but flipped. Many people left from OpenAI to Anthropic, I think all original founders of Anthropic are still with Anthropic.

    We have seen the same things in Web2 and Web3, this is not at all unique to AI companies! How many people at HN went to work for Facebook, or Google etc? The “don’t be evil” motto is nice but when you have to compete, you often end up doing the things you think you didn’t have to do.

    With Web2 the stakes were: “centrally controlling speech”. With Web3 the stakes were: “people lose programmable money they gambled with”. With AI the stakes are much bigger. This isn’t about a company being moral or not. This is about incentives of the broader ecosystem, and how the products are designed.

    As long as people (like Kevin O’Leary) say “but are you going to just let China pull ahead of us” we’re all going to be in a race (to the bottom, for a lot of affected people). It isn’t about OpenAI or Anthropic specifically. Hate the game, not the player.

    It doesn’t have to be this way. The game can change: https://safebots.ai/singularity.html