Comment by nettlin

1 day ago

They just added more details:

> Indicators of compromise (IOCs)

> Our investigation has revealed that the incident originated from a third-party AI tool whose Google Workspace OAuth app was the subject of a broader compromise, potentially affecting hundreds of its users across many organizations.

> We are publishing the following IOC to support the wider community in the investigation and vetting of potential malicious activity in their environments. We recommend that Google Workspace Administrators and Google Account owners check for usage of this app immediately.

> OAuth App: 110671459871-30f1spbu0hptbs60cb4vsmv79i7bbvqj.apps.googleusercontent.com

https://vercel.com/kb/bulletin/vercel-april-2026-security-in...

https://x.com/rauchg/status/2045995362499076169

> A Vercel employee got compromised via the breach of an AI platform customer called http://Context.ai that he was using.

> Through a series of maneuvers that escalated from our colleague’s compromised Vercel Google Workspace account, the attacker got further access to Vercel environments.

> We do have a capability however to designate environment variables as “non-sensitive”. Unfortunately, the attacker got further access through their enumeration.

> We believe the attacking group to be highly sophisticated and, I strongly suspect, significantly accelerated by AI. They moved with surprising velocity and in-depth understanding of Vercel.

Still no email blast from Vercel alerting users, which is concerning.

  • > We believe the attacking group to be highly sophisticated and, I strongly suspect, significantly accelerated by AI. They moved with surprising velocity and in-depth understanding of Vercel.

    Blame it on AI ... trust me... it would have never happened if it wasn't for AI.

  • > We believe the attacking group to be highly sophisticated and, I strongly suspect, significantly accelerated by AI.

    Reads like the script of a hacker scene in CSI. "Quick, their mainframe is adapting faster than I can hack it. They must have a backdoor using AI gifs. Bleep bleep".

  • > Still no email blast from Vercel alerting users, which is concerning.

    On the one hand, I get that it's a Sunday, and the CEO can't just write a mass email without approval from legal or other comms teams.

    But on the other hand... It's Sunday. Unless you're tuned-in to social media over the weekend, your main provider could be undergoing a meltdown while you are completely unaware. Many higher-up folks check company email over the weekend, but if they're traveling or relaxing, social media might be the furthest thing from their mind. It really bites that this is the only way to get critical information.

    • > On the one hand, I get that it's a Sunday, and the CEO can't just write a mass email without approval from legal or other comms teams

      This is not how things work. In a crisis like this there is a war room with all stakeholders present. Doesn’t matter if it’s Sunday or 3am or Christmas.

      And for this company specifically, Guillermo is not one to defer to comms or legal.

      2 replies →

    • > the CEO can't just write a mass email without approval from legal or other comms teams.

      They can be brought in to do their job on a Sunday for an event of this relevance. They can always take next Friday off or something.

    • Has anyone actually gotten an email from Vercel confirming their secrets were accessed? Right now we're all operating under the hope (?) that since we haven't (yet?) gotten an email, we're not completely hosed.

      4 replies →

    • I'm going down with the ship over on X.com the Everything App. There's a parcel of very important tech people that are running some playbook where posting to X.com is sufficient enough to be unimpeachable on communication, despite its rather beleaguered state and traffic.

  • > an AI platform customer called http://Context.ai that he was using

    Hmm? Who is the customer in this relationship? Is Vercel using a service provided by Context.ai which is hosted on Vercel?

  • Production network control plane must be completely isolated from the internet with a separate computer for each. The design I like best is admins have dedicated admin workstations that only ever connect to the admin network, corporate workstations, and you only ever connect to the internet from ephemeral VMs connected via RDP or similar protocol.

The actual app name would be good to have. Understandable they don’t want to throw them under the bus but it’s just delaying taking action by not revealing what app/service this was.

I don’t understand why they can’t just directly name the responsible app as it will come out eventually.

Idk exactly how to articulate my thoughts here, perhaps someone can chime in and help.

This feels like a natural consequence of the direction web development has been going for the last decade, where it's normalised to wire up many third party solutions together rather than building from more stable foundations. So many moving parts, so many potential points of failure, and as this incident has shown, you are only as secure as your weakest link. Putting your business in the hands of a third party AI tool (which is surely vibe-coded) carries risks.

Is this the direction we want to continue in? Is it really necessary? How much more complex do things need to be before we course-correct?

  • This isn't a web development concept. It's the unix philosophy of "write programs that do one thing and do it well" and interconnect them, being taken to the extremes that were never intended.

    We need a different hosting model.

    • Just throwing it out there - the Unix way to write software is often revered. But ideas about how to write software that came from the 1970s at Bell Labs might not be the best ideas for writing software for the modern web.

      Instead of "programs that do one thing and do it well", "write programs which are designed to be used together" and "write programs to handle text streams", I might go with a foundational philosophy like "write programs that are do not trust the user or the admin" because in applications connected to the internet, both groups often make mistakes or are malicious. Also something like "write programs that are strict on which inputs they accept" because a lot of input is malicious.

      4 replies →

    • > We need a different hosting model.

      There really isn't an option here, IMO.

      1. Somebody does it

      2. You do it

      Much happier doing it myself tbh.

      1 reply →

    • It's not a hosting model, it's a fundamental failure of software design and systems engineering/architecture.

      Imagine if cars were developed like websites, with your brakes depending on a live connection to a 3rd party plugin on a website. Insanity, right? But not for web businesses people depend on for privacy, security, finances, transportation, healthcare, etc.

      When the company's brakes go out today, we all just shrug, watch the car crash, then pick up the pieces and continue like it's normal. I have yet to hear a single CEO issue an ultimatum that the OWASP Top 10 (just an example) will be prevented by X date. Because they don't really care. They'll only lose a few customers and everyone else will shrug and keep using them. If we vote with our dollars, we've voted to let it continue.