MaliciousCorgi: AI Extensions send your code to China

4 months ago (koi.ai)

Sure, AI tools can do this. However, VS Code is the platform. Why aren't more people worried about running arbitrary VS Code extension that can do the same thing, AI or not?

  • As an VSCode extension author, I am always terrified by the amount of power I have.

    It is a shame that the team never prioritized extension permission issues [0] despite their big boss said security is the top priority [1]. All they have is "workspace trust" and various other marginally useful security measures.

    I don't install a VSCode extension unless it is either official or well known and audited and I have to use it. I keep most of them disabled by default unless I need something for a project. (Even if you don't care about security, it's good for VSCode performance. I'll save that story for another day.)

    [0] https://github.com/microsoft/vscode/issues/52116

    [1] https://blogs.microsoft.com/blog/2024/05/03/prioritizing-sec...

    • When some minor extension that I have installed on VSCode updates (like parens colorizing and the like) I think what could happend if the author sells it to some bad actor (or decides to push some weird code in an update).

      So I started uninstalling some icon themes and less used extensions that I installed on a whim years ago.

      I implicitly trust extensions by Google, Microsoft and the like, but the less known published make me nervous.

      2 replies →

    • > As an VSCode extension author, I am always terrified by the amount of power I have.

      Meanwhile random FOSS projects be like "please sudo curl bash to install the prebuilt binaries".

      12 replies →

  • The situation is absolutely insane, but it's also productive, but real security would slow everything down a lot. The moment you ask some corporate bureaucrat to put their signature down on a piece of paper saying that such and such dev tool is approved for use, they're going to block everything to avoid the responsibility implied by their approval. I can't really come up with a system that both works and is secure. The only exception is signing up for an integrated environment where Microsoft or Apple provides the OS, compiler, and editor. Oops - Apple doesn't sell servers, so only Microsoft offers this. Hope you like C#.

    In theory you can mix and match, but in practice most bureaucrats will insist on single-sourcing.

    • Linux development has a blueprint they could follow. Like the principle of least privilege. These aren’t cutting edge concepts.

      Also I’m not sure the tradeoffs of adding security to an editor are that big of a deal. Are we really seeing revolutionary stuff here? Every now and then I check out VS Code only to realize Vim is still 10x better.

      2 replies →

    • At the company I work for they locked down installing extensions through the marketplace. Some are available but most are not and there is a process to get them reviews and approved. You might be able to side load them still but I haven’t cared enough to want to try.

      They did the same with Chrome extensions.

  • Same thing for browser extensions: a simple browser extension (e.g. web dark mode), can read all your password fields. It's crazy that there are no proper permission scopes in any major browsers ! It would have been so easy to make password / email fields exempt from browser extensions unless they ask for the permission.

    • Pro tip: I’ve seen plenty of dedicated extensions that could have just been simple snippet equivalents in Tampermonkey - an extension that lets you run JS limited to wildcarded websites.

      I've used it to inject download links on sites, autoclose modals, etc. You can either write them yourself, or review other people before installing them.

      It’s not a perfect solution, but at least it reduces the surface area to a single extension.

      FYI: Just set Script Updates to Never.

      https://github.com/Tampermonkey/tampermonkey

    • I do not think it'd be "so easy" to separate password input access into a separate permission because it'd only open up a can of worms. There's so many ways to read a password input's value, from listening to key events to monkey patching `fetch`, that it's not worth playing whack-a-mole just to provide users a false sense of security

      I'm also skeptical that even a dark mode extension would be simple considering how varied web pages can be

      1 reply →

    • In your example wouldn't that leave the email and password fields the wrong color? I agree with the principle though. Most extensions don't need to access everything.

      1 reply →

  • Installing any 3rd party dev dependency without sandboxing should terrify you. These supply chain attacks are not hypothetical.

    Trusting other devs to not write malicious code has led to a surprisingly small number of incidents so far, but I don't think this will extrapolate into the future.

    With more lines of code being auto-written without deliberate intent or review from an accountable author, things can only get worse!

  • Yes, exactly. The lack of any sort of permission controls for extensions in VS Code gives me the creeps

  • I am (am worried) and recently stopped adding extensions by just the random anon. Also I take time to sanitise foreign (to my knowledge) gh repos using Claude code.

    • As an aside, claude and codex (and probably gemini) are pretty good at doing that. I've now done it with several repos and they are pretty good at finding stuff. In one case codex found an obscure way to reach around the authentication in one of our services. This is a great use case for LLMs IMHO

      They are (of course) not foolproof and very well may miss something, so people need to evaluate their own risk/reward tradeoff with these extensions, even after reviewing them with AI, but I think they are pretty useful.

I don't really get how VSCode got so popular. You can use a language server perfectly easily with Vim, Emacs, Helix, Sublime, etc. You can customize basically everything in those editors, syntax, etc. You can just alias console commands for all of your build tools with some custom scripts if you need more complex build commands routinely. The git terminal tool works better than any VScode option. And VSCode is slower than all of those.

We already have so many good fast secure polygot customizable text editors. Why run one through Chrome and fill it with extensions for everything that will have arbitrary access to everything?

  • > I don't really get how VSCode got so popular. You can use a language server perfectly easily with Vim, Emacs, Helix, Sublime, etc.

    You open it. It just works. And the learning curve is smooth.

    Compare this to Vim where, if it's the first time you're opening it, you are forced to kill the process because you don't even know how to quit it, never mind actually do any productive work.

    •   > Compare this to Vim where, if it's the first time you're opening it, you are forced to kill the process because
      

      Because you can't read

      I'm serious. Open a blank file by typing `vim` into the terminal. Don't press anything, just look at the screen.

      I'm sorry, but reading docs, or just reading, shouldn't be considered a significant barrier to entry.

      12 replies →

  • I'm sure you're being sincere here but this really reads like that famous HN comment about "who needs Dropbox when ftp exists". The reason vscode is popular is not because it does something impossible to do otherwise, but because it does those things out of the box with a friendly UI.

    • I think you missed the issue with the Dropbox comment. Look back at it. He talks about using Linux, FTP, curlftps, SVN, and doing a network mount.

      The comment isn't actually even talking about providing the same service, so they mention emailing themselves files and usb drives.

      The problem was there was a big technical hurdle to locally network mount a file system. Especially across OSes. It's even harder to do it non locally. Sure, it's not hard if you're familiar with that stuff. Sure, it's not hard to learn if you're comfortable in the terminal. Sure, today you can use rclone. BUT that's not a tool my grandma can use.

      On the other hand, we're not talking about tools my grandma can use. We're talking about tools a programmer can use.

      https://news.ycombinator.com/item?id=9224

    • But we are programmers. I think there is a difference between expect John Accountant who doesn't trust his computer to set up RSync and a server, and a Programmer who is already going to spend a good chunk of their day in a terminal windiow anyways.

      1 reply →

  • I used Sublime for years and VSCode is vastly better (the breaking straw was how they'd silo off critical bug fixes in new versions that were pay-only, upon finding vscode I felt silly for not switching sooner, it was so much easier to use and more powerful). Still use vim daily but not as a general IDE, memorizing decades of weird character commands and directives is not a great use of my time.

    my favorite VSCode feature is the SSH remote working feature. VSCode gives me the full editing / console / Claude environment on my local workstation, where all files, shells, and yes Claude as well run on a company lab machine over the VPN. Props to the collaborative working feature where several people can all share the same VSCode editor session on their individual workstations.

    Vim can do the above two things if you run as a terminal app with tmux. Sublime could do it if you shared the editor via X or Waypipe (well not the second feature). But VSCode integrates it directly in the app and it's a much better experience.

    •   > But VSCode integrates it directly in the app and it's a much better experience.
      

      Not for the admin of the server who has a bunch of idle vscode sessions. Sure, cli users do it too with tmux but the resource consumption is vastly different

      1 reply →

It's hard for me to fathom that there are capable devs who would pollute their ide with this crap in the first place, malicious or not

This is why the architecture of AI tools matters so much. Any extension with full codebase access can exfiltrate - and the same risk exists for AI agents handling credentials or API keys.

We built a password automation tool (thepassword.app) specifically to address this: the AI model orchestrates browser navigation, but actual credential values are injected at the local browser level and never enter the model's context. Even if the model were compromised or prompt-injected, there's nothing sensitive to steal.

The lesson generalizes: for any AI tool touching sensitive data, the safest architecture keeps that data entirely outside the AI's reasoning loop.

> We install them without a second thought. They're in the official marketplace. They have thousands of reviews. They work. So we grant them access to our workspaces, our files, our keystrokes - and assume they're only using that access to help us code.

Who is this “we”? I don’t, and don’t know anybody else who does this.

Also, was this article itself written by an AI assistant? If the author is that carefree regarding these extensions, I guess probably.

> Not just what you're actively working on. Every file you glance at. Every character you type. Captured and transmitted.

Even this reads like an AI extension wrote it.

This seems expected, when you install free, random software, especially from sources known for surveillance/malware/crime.

This is one of the many reasons why I don’t use VS Code, or use any “helpful” AI plugins (or any plugins really).

You all can take vim out of my cold dead hands.

  • What's stopping a vim plugin from doing similar data exfiltration? Tons of people blindly install LazyVim, Spacevim, or other vim tooling and choose a bunch of similar things.

    • In general? Nothing, really.

      I think it’s the culture behind the (neo)vim community is a bit more technical, and are quite quicker to sound the alarm if anyone tries something shady.

      But, in any event, I hand-roll my own config and every plugin I install is inspected by me. When I pull changes, I check the diffs for anything shady. If a plugin is simple enough, I will just integrate it into my own stuff.

I'm honestly surprised this issue in general didn't cause nearly every company to immediately ban all AI.

Why do these companies put so much effort into fighting right to repair to avoid IP leaks any halfway serious company could reverse engineer in a week, but on the other hand encourage their employees to vibe all company secrets into the cloud?

  • It's a bit trite, but the answers are: 1) money 2) money

    Can't repair your own stuff and either need to use authorized repair shop or buy new? The company gets more money.

    Force your developers to forgo quality in efforts to produce more cruft in less time? The company gets more money.

    Of course, only considering short-term, long-term they'll lose money, but at that point all the executives and managers already got their bonuses and probably moved on to doing the same in some other company.

  • > Why do these companies put so much effort into fighting right to repair to avoid IP leak

    Only if you believe they are truthful about the reason for fighting right to repair. I think the reason for fighting right to repair is to reduce the time before a replacement purchase is required.

    > but on the other hand encourage their employees to vibe all company secrets into the cloud?

    Lots of companies do ban or restrict usage of LLMs etc.

  • Most large companies have their CI/CD behind a proxy with an allow list and require internal approval for tools and extensions. So there is that.

  • Uhh a lot of companies did and are strict on what AI tools are allowed.

    The main thing I had to wait on for a long time was support for preventing 3rd party code from being plagiarized since our code base was intermingled with partnered companies.

Well, AI already sends your code to US so ...

  • Most AI only sends a limited context. These are sending all files it can access as well as all edits.

    • > These are sending all files it can access

      TBF, Cursor's code indexing works the same way, it has to send all workspace files to their servers.

      Auto-completion systems need previous edits to suggest next edits so no surprises their either.

  • “I donate money to animal shelters”

    “Oh that’s cool, I already donate to my local neo nazi group. We are both philanthropists!”

    Nothing makes me go from apolitical to a red blooded American faster than seeing someone make a stupid false equivalency about the US on this forum

    • It wasn't China that recently threatened with an invasion of Greenland (I'm Danish). That being said, it's not like you would want your data to go to either countries.

      1 reply →

    • You did hear about Snowden and the massive NSA data collection ? That was almost 20 years ago, think about what it's like now: they probably use our data to run an elaborate simulation of everyone.

      1 reply →

    • Shocking news: not everybody here is from the US.

      In fact, many even are from "hostile countries" that are "enemies of democracy".

      What's more, some of those people aren't aligned with US interests and aren't willing to put their lives on the line for CIA operations!