Comment by junon
3 months ago
Hi, yep I got pwned. Sorry everyone, very embarrassing.
More info:
- https://github.com/chalk/chalk/issues/656
- https://github.com/debug-js/debug/issues/1005#issuecomment-3...
Affected packages (at least the ones I know of):
- ansi-styles@6.2.2
- debug@4.4.2 (appears to have been yanked as of 8 Sep 18:09 CEST)
- chalk@5.6.1
- supports-color@10.2.1
- strip-ansi@7.1.1
- ansi-regex@6.2.1
- wrap-ansi@9.0.1
- color-convert@3.1.1
- color-name@2.0.1
- is-arrayish@0.3.3
- slice-ansi@7.1.1
- color@5.0.1
- color-string@2.1.1
- simple-swizzle@0.2.3
- supports-hyperlinks@4.1.1
- has-ansi@6.0.1
- chalk-template@1.1.1
- backslash@0.2.1
It looks and feels a bit like a targeted attack.
Will try to keep this comment updated as long as I can before the edit expires.
---
Chalk has been published over. The others remain compromised (8 Sep 17:50 CEST).
NPM has yet to get back to me. My NPM account is entirely unreachable; forgot password system does not work. I have no recourse right now but to wait.
Email came from support at npmjs dot help.
Looked legitimate at first glance. Not making excuses, just had a long week and a panicky morning and was just trying to knock something off my list of to-dos. Made the mistake of clicking the link instead of going directly to the site like I normally would (since I was mobile).
Just NPM is affected. Updates to be posted to the `/debug-js` link above.
Again, I'm so sorry.
We also caught this right away at Socket,
https://socket.dev/blog/npm-author-qix-compromised-in-major-...
While it sucks that this happened, the good thing is that the ecosystem mobilized quickly. I think these sorts of incidents really show why package scanning is essential for securing open source package repositories.
So how do you detect these attacks?
We use a mix of static analysis and AI. Flagged packages are escalated to a human review team. If we catch a malicious package, we notify our users, block installation and report them to the upstream package registries. Suspected malicious packages that have not yet been reviewed by a human are blocked for our users, but we don't try to get them removed until after they have been triaged by a human.
In this incident, we detected the packages quickly, reported them, and they were taken down shortly after. Given how high profile the attack was we also published an analysis soon after, as did others in the ecosystem.
We try to be transparent with how Socket work. We've published the details of our systems in several papers, and I've also given a few talks on how our malware scanner works at various conferences:
* https://arxiv.org/html/2403.12196v2
* https://www.youtube.com/watch?v=cxJPiMwoIyY
13 replies →
AI based code review with escalation to a human
6 replies →
[flagged]
Apparently it found this attack more or less immediately.
It seems strange to attack a service like this right after it actively helped keep people safe from malware. I'm sure its not perfect, but it sounds like they deserve to take a victory lap.
2 replies →
You could at least offer some kind of substantive criticism of the tool (“socket”).
7 replies →
For those interested, points associated with this post spiked to at least 4 then dropped back to one. Take of that what you will.
Just want to agree with everyone who is thanking you for owning up (and so quickly). Got phished once while drunk in college (a long time ago), could have been anyone. NPM being slowish to get back to you is a bit surprising, though. Seems like that would only make attacks more lucrative.
Can happen to anyone… who doesn’t use password manager autofill and unphishable 2FA like passkeys.
Most people who get phished aren’t using password managers, or they would notice that the autofill doesn’t work because the domain is wrong.
Additionally, TOTP 2FA (numeric codes) are phishable; stop using them when U2F/WebAuthn/passkeys are available.
I have never been phished because I follow best practices. Most people don’t.
I use a password manager. I was mobile, the autofill stuff isn't installed as I don't use it often on my phone.
In 15 years of maintaining OSS, I've never been pwned, phished, or anything of the sort.
Thank you for your input :)
14 replies →
You can use password manager autofill and hardware 2fa and still get phished. All it takes is you rushing, not paying attention, clicking on a link, and logging in (been caught by my own security team doing this). Yes, in an ideal world you're going to be 100% perfect. The world is not ideal, unfortunately. I don't have a solution, but demanding humans behave perfectly in order to remain secure is not a reasonable ask.
I also use WebAuthn where possible but wouldn’t be so cocky. The most likely reason why we haven’t been phished because we haven’t been targeted by a sophisticated attacker.
One side note: most systems make it hard to completely rely on WebAuthn. As long as other options are available, you are likely vulnerable to an attack. It’s often easier than it should be to get a vendor to reset MFA, even for security companies.
5 replies →
> I have never been phished because I follow best practices. Most people don’t.
You forgot to mention that you are both highly skilled and practiced at phishing yourself... don't you think that helps too?
in general npm does a not-too-great job with these things
Remember, NPM stands for Now Part of Microsoft!
(Microsoft owns GitHub, which owns NPM.)
2 replies →
[dead]
Hey, no problem, man. You do a lot for the community, and it's not all your fault. We learn from our mistakes. I was thinking of having a public fake profile to avoid this type of attack, but I'm not sure how it would work on the git tracking capabilities. Probably keeo it only internally for you&NPM ( the real one ) and have some fake ones open for public but not sure, just an obfuscated idea. Thanks for taking the responsibility and working in fixing ASAP. God bless you.
Unfortunately wouldn't have helped. They skimmed my npm-only address directly from the public endpoint.
Wow, that's actually kinda genius not gonna lie. Honestly, I would love seeing some 2fa or some other way to prevent pwning. Maybe having a sign up with google with all of its flaws still might make sense given how it might be 2fa.
But google comes with its own privacy nightmares.
Tbh, it's not your fault per se; everybody can fall for phishing emails. The issue, IMO, lies with npmjs which publishes to everyone all at the same time. A delayed publish that allows parties like Aikido and co to scan for suspicious package uploads first (e.g. big changes in patch releases, obfuscated code, code that intercepts HTTP calls, etc), and a direct flagging system at NPM and / or Github would already be an improvement.
Being able to sign releases would help, too. I would happily have that enabled since I'm always publishing from one place.
Wouldn't they have been able to change your key if they had compromised your entire npm account?
Also, junon.support++ – big thanks for being clear about all this.
5 replies →
Provenance can be added to NPM https://docs.npmjs.com/generating-provenance-statements
So if the hacker did an npm publish from local it would show up.
Yeah; I wish provenance was more widely used. I think about this a lot for mobile apps. If you take an opensource iOS app like signal, you can read the source code on github. But there's actually no guarantee that the code on github corresponds in any way to the app I download from the app store.
With nodejs packages, I can open up node_modules and read the code. But packages get a chance to run arbitrary code on your computer after installation. By the time you can read the source code, it may be too late.
Thanks for sounding the alarm. I've sent an abuse email to porkbun to hopefully get the domain taken down.
Thank you, I appreciate it! I did so as well and even called their support line to have them escalate it. Hopefully they'll treat this as an urgent thing; I'd imagine I'm far from the only one getting these.
It's down, so there's some good news. Probably worth submitting to IC3 as well.
Yo, someone at npm needs to unpublish simple-swizzle@0.2.3 IMMEDIATELY. It’s still actively compromised.
It's been almost two hours without a single email back from npm. I am sitting here struggling to figure out what to do to fix any of this. The packages that have Sindre as a co-publisher have been published over but even he isn't able to yank the malicious versions AFAIU.
If there's any ideas on what I should be doing, I'm all ears.
EDIT: I've heard back, they said they're aware and are on it, but no further details.
NPM is a Github company and when there was a relatively serious attack in Github Actions a while back there was also pretty much zero response from them.
Github is SOC2 compliant, but that of course means nothing really.
They have yanked the bad version of simple-swizzle by now, which was the last of the packages that I was tracking.
It took them quite a long time to do so.
My god. The npm team should urgently review their internal processes. These two hours of neglect will cost a lot of money downstream. At this stage, they act nothing short of irresponsible.
I haven't published anything to npm in over a decade. But if you still have access to git, a cli, or a browser where the login is cached and you can access it, you should do so and either take the code down or intentionally sabotage/break it.
I can not find the package anymore. I think someone did it already.
Thank you for your service.
Please take care and see this as things that happen and not your own personal failure.
Hey, you're doing an exemplary response, transparent and fast, in what must be a very stressful situation!
I figure you aren't about to get fooled by phishing anytime soon, but based on some of your remarks and remarks of others, a PSA:
TRUSTING YOUR OWN SENSES to "check" that a domain is right, or an email is right, or the wording has some urgency or whatever is BOUND TO FAIL often enough.
I don't understand how most of the anti-phishing advice focuses on that, it's useless to borderline counter-productive.
What really helps against phishing :
1. NEVER EVER login from an email link. EVER. There are enough legit and phishing emails asking you to do this that it's basically impossible to tell one from the other. The only way to win is to not try.
2. U2F/Webauthn key as second factor is phishing-proof. TOTP is not.
That is all there is. Any other method, any other "indicator" helps but is error-prone, which means someone somewhere will get phished eventually. Particularly if stressed, tired, or in a hurry. It just happened to be you this time.
Good luck and well done again on the response!
> NEVER EVER login from an email link. EVER
Login using one off email links (instead of username + password) is increasingly common which means its the only option.
In that case
1. You just requested it, I'm not saying to never click link on transactional emails you requested. You still need to click on those verify email links
2. It replaces entering your password, so you're not entering your password on a link from an email, which is the very wrong thing.
At least you've requested that email, to be able to login. The timing chance for a phishing mail to come here and there is insignificant. OP is referring to communications that are one way street, the (pseudo) organisation to you.
6 replies →
Or you know, get a password manager like the rest of us. If your password manager doesn't show the usual autofill, since the domain is different than it should, take a step back and validate everything before moving on.
Have the TOTP in the same/another password manager (after considering the tradeoffs) and that can also not be entered unless the domain is right :)
I feel like it's extremely common for the autofill to not work for various reasons even when you aren't being phished. I have to manually select the site to fill fairly often, especially inside apps where the password manager doesn't seem to match the app to the website password.
Passkeys seem like the best solution here where you physically can not fall for a phishing attack.
5 replies →
I mostly agree and I do use one.
You only need read the whole thread however to see reasons why this would sometimes not be enough: sometimes the password manager does not auto-fill, so the user can think it's one of those cases, or they're on mobile and they don't have the extension there, or...
As a matter of fact, he does use one, that didn't save him, see: https://news.ycombinator.com/item?id=45175125
14 replies →
I wish it's that easy. 1Password autofill on Android Chrome broke for me a month ago. Installed all updates, checked settings, still nothing. Back to phishing prone copy paste.
Could happen to any of us. Thanks for reacting so quickly!!
Absolutely best response here.
Folks from multi-billion dollar companies with multimillion dollar packages should learn a few things from this response.
Didn't your password manager notice that npmjs dot help was not a legit domain and avoid auto-filling there?
https://news.ycombinator.com/item?id=45175125
Thank you for the swift and candid response, this has to suck. :/
> The author appears to have deleted most of the compromised package before losing access to his account. At the time of writing, the package simple-swizzle is still compromised.
Is this quote from TFA incorrect, since npm hasn’t yanked anything yet?
Quote is probably added recently. Not entirely correct as I have not regained access; nothing happening to the packages is of my own doing.
npm does appear to have yanked a few, slowly, but I still don't have any insight as to what they're doing exactly.
The fact that NPMs entire ecosystem relies on this not happening regularly is very scary.
I’m extremely security conscious and that phishing email could have easily gotten me. All it takes is one slip up. Tired, stressed, distracted. Bokm, compromised
Could happen to anyone, many thanks for addressing this quickly.
I hate that kind of email when sent out legitimately. Google does this crap all the time pretty much conditioning their customers to click those links. And if you're really lucky it's from some subdomain they never bothered advertising as legit.
Great of you to own up to it.
Atlassian and MS are terrible for making email notifications that are really hard to distinguish from phishing emails. Using hard to identify undocumented random domains in long redirect chains, obfuscating links etc etc.
I’ve started ignoring these types of emails and wait to do any sort of credentials reset until I get an alert when I log in (or try to) for just this reason.
Thank you for being quick and upfront about this!
What did the phishing email say that made you click and login?
That it had been more than 12 months since last updating them. Npm has done outreach before about doing security changes/enhancements in the past so this didn't really catch me.
Screenshot here: https://imgur.com/a/q8s235k
@everyone in the industry, everywhere:
Urgency is poison.
Please, please put a foot in the door whenever you see anyone trying to push this kind of sh*t on your users. Make one month's advance notice the golden standard.
I see this pattern in scam mail (including physical) all the time: stamp an unreasonably short notice and expect the mark to panic. This scam works - and this is why legit companies that try this "in good faith" should be shamed for doing it.
Actual alerts: just notify. Take immediate, preventive, but non-destructive action, and help the user figure out how to right it - on their own terms.
22 replies →
Can you post full message headers somewhere? It'd be interesting which MTA was involved in delivery from the sender's side.
14 replies →
Thanks for sharing, I've created an OTX entry for this: https://otx.alienvault.com/pulse/68bf031ee0452072533deee6
1 reply →
Perfect example of why habituating users to renewing credentials (typically password expiration) is a terrible practice.
5 replies →
Yikes, looks legit. Curious what are the destination addresses? Would like to monitor them to see how much coin they are stealing.
9 replies →
And then what happens when you click the link? Wouldn't your password manager fail to auto fill your details?
1 reply →
That green checkmark ... what application is this?
2 replies →
I am not very sophisticated npm user on MacOS, but I installed bunch of packages for Claude Code development. How do we check if computer has a problem?
Do we just run:
npm list -g #for global installs
npm list #for local installs
And check if any packages appear that are on the above list?
Thanks!
How I do it is, run npm list --all then check the completely dependency tree to find out if anywhere I am using the vulnerable package.
> Made the mistake of clicking the link instead of going directly to the site like I normally would (since I was mobile).
Does anyone know how this attack works? Is it a CSRF against npmjs.com?
That was the low-tech part of their attack, and was my fault - both for clicking on it and for my phrasing.
It wasn't a single-click attack, sorry for the confusion. I logged into their fake site with a TOTP code.
This is a clear example that this can happen to anyone.
Sorry for what you're going through.
1 reply →
Fake site.
You login with your credentials, the attacker logins to the real site.
You get an SMS with a one time code from the real site and input it to the fake site.
The attacker takes the code andc finishes the login to the real site.
Probably just a fake site.
Hey, new dev here. Sorry if this is a common knowledge and I am asking a stupid question. How does you getting phished affect these NPM packages? aren't these handled by NPM or the developers of them?
The guy is actually the maintainer of those packages. So whoever got his credentials became able to perform releases on those packages. NPM itself does not build any package, it's just a place where people can publish stuff
OP is the developer & maintainer of the affected packages, so the attacker was able to use their phished credentials to upload compromised versions to NPM.
oh! understood. thanks.
You're doing what you can, it's not easy. Thanks for handling this so well.
`error-ex` 1.3.3, already removed from npm https://github.com/Qix-/node-error-ex/issues/17
Happens to the best of people. Appreciate you’re fast and open response.
The 2FA/TOTP security theater was partly to blame for this.
How so? Has the author mentioned somewhere that he was tricked into providing 2FA codes / had any sort of 2FA enabled at all?
A spearphishing email telling them they had to update their 2FA was the vector.
Thanks for leaving a transparent response with what happened, how you responded, what you're doing next, and concisely taking accountability Great work!
Insanely well crafted phishing, godspeed man.
Thanks Josh, appreciate it <3
I'm sorry that you're having to go through this. Good luck sorting out your account access.
I actually got hit by something that sounds very similar back in July. I was saved by my DNS settings where "npNjs dot com" wound up on a blocklist. I might be paranoid, but it felt targeted and was of a higher level of believability than I'd seen before.
I also more recently received another email asking for an academic interview about "understanding why popular packages wouldn't have been published in a while" that felt like elicitation or an attempt to get publishing access.
Sadly both of the original emails are now deleted so I don't have the exact details anymore, but stay safe out there everyone.
Thanks for your response. But this does call for preventing a single point of failure for security.
maybe you should work with feross to make a website-api that simply gives you a "true/false" on "can I safely update my dependencies right now" that gives an outofband way to mark the current or all versions thereof, of compromised packages.
So by "Just NPM is affected" does that mean yarn is unaffected?
No, anything that connects to npm as an authoritative source for packages. Yarn, pnpm, and npm clients all do.
Thank god I misspelled "npm run strat"! Might have been owned.
insanely well-crafted. i mean, it's something bad that happened but one must recognise the wit of this attack.
mistakes happen. owning them doesn't always happen, so well done.
phishing is too easy. so easy that I don't think the completely unchecked growth of ecosystems like NPM can continue. metastasis is not healthy. there are too many maintainers writing too many packages that too many others rely on.
we're only human mate, great job responding to it!
thanks for your efforts!
be careful!
Sorry to be dumb, but can you expand a bit on "2FA reset email..." so the rest of us know what not to do?
Ignore anything coming from npm you didn't expect. Don't click links, go to the website directly and address it there. That's what I should have done, and didn't because I was in a rush.
Don't do security things when you're not fully awake, too. Lesson learned.
The email was a "2FA update" email telling me it's been 12 months since I updated 2FA. That should have been a red flag but I've seen similarly dumb things coming from well-intentioned sites before. Since npm has historically been in contact about new security enhancements, this didn't smell particularly unbelievable to my nose.
The email went to the npm-specific inbox, which is another way I can verify them. That address can be queried publicly but I don't generally count on spammers to find that one but instead look at git addresses etc
The domain name was `npmjs dot help` which obviously should have caught my eye, and would have if I was a bit more awake.
The actual in-email link matched what I'd expect on npm's actual site, too.
I'm still trying to work out exactly how they got access. They didn't technically get a real 2FA code from the actual, I don't believe. EDIT: Yeah they did, nevermind. Was a TOTP proxy attack, or whatever you'd call it.
Will post a post-mortem when everything is said and done.
I see (I think): they tricked you into entering a TOTP code into their site, which they then proxied to the real names, thereby authenticating as your account. Is that correct?
22 replies →
Did they also phish the login password after clicking the link or did they already have it?
5 replies →
> because I was in a rush
That's how they get you.
Using a security key as 2FA instead of TOTP would have prevented this attack, right?
If you maintain popular open source packages for the love of God get yourself a couple of security keys.
1 reply →
> The domain name was `npmjs dot help` which obviously should have caught my eye, and would have if I was a bit more awake.
It's a good thing the WebPKI cartel mostly did away with EV certs.... these days any old cert where only the SAN matches the domain and your browser gives a warm fuzzy "you're secure!"
4 replies →
> so the rest of us know what not to do?
Can't really tell you what not to do, but if you're not already using a password manager so you can easily avoid phishing scams, I really recommend you to look into starting doing so.
In the case of this attack, if you had a password manager and ended up on a domain that looks like the real one, but isn't, you'd notice something is amiss when your password manager cannot find any existing passwords for the current website, and then you'd take a really close look at the domain to confirm before moving forward.
After nearly being phished once (only having a confirmation email save me) I've taken to being extra vigilant if I don't get a password entry suggestion from my password manager. It means I need to be extremely damn sure I'm on a domain that is controlled by the same entity my account is with. So far I haven't had another incident like that and I hope to keep it that way.
This isn’t exactly true. My password manager fails to recognise the domain I’m on, all the time. I have to go search for it and then copy/paste it in.
That being said, if you’re making login pages: please, for the love of god, test them with multiple password managers. Oh, and make sure they also work correctly with the browser’s autotranslation. Don’t rely on the label to make form submission decisions ... please.
7 replies →
Hang in there buddy. These things happen.
man. anyone and everyone can get fished in a targeted attack. good luck on the cleanup and thanks for being forward about it.
want to stress everyone it can happen to. no one has perfect opsec or tradecraft as a 1 man show. its simply not possible. only luck gets one through and that often enough runs out.
Not your fault. Thanks for posting and being proactive about fixing the problem. It could happen to anyone.
And because it could happen to anyone that we should be doing a better job using AI models for defense. If ordinary people reading a link target URL can see it as suspicious, a model probably can too. We should be plumbing all our emails through privacy-preserving models to detect things like this. The old family of vulnerability scanners isn't working.