It's insane the AI has been provided the tooling to send emails to arbitrary addresses like that. Like, getting it to send a 2FA code at a user's request is one thing. But it should only be able to "hit a button" to send a 2FA email to the address attached to the account, all run with hand-written code. It shouldn't have access to the 2FA code itself, or the message subject, or body, or the recipient address, etc.
My impression is that AI didn't replace static code in this place; it replaced a person, who (hopefully) would have been suspicious about sending an account recovery code for e.g. "obamawhitehouse" to e.g. "bscurtu.alfamm.ro@gmail.com"
This is not true. Well, it kinda is, but nobody will be stupid enough to hand-code an account recovery where you get to type any email address.
The reason it worked there is that the designers of the system didn't anticipate that the AI will agree to accept any email (maybe they even put guardrails against it in the system prompt, we don't know). It's more like social engineering than bad-security-code, except that like the sibling comment said an actual human will probably not approve that.
> This exact same flow could have been…statically coded.
But had never been until it was wrapped in a chatbot. It’s just about unheard of for a major site in the modern era, isn’t it? I think the AI factor is essentially essential. All but.
The reason all these meticulously designed flows have been done away with is because some manager believes that AI is omniscient and can just replace it all.
Like, flagging VPN endpoints is bread and butter for this kind of thing and must already exist. But it's been bypassed
An email address is making its way from a publicly available LLM prompt input to a sensitive email's recipient address. That's the problem I'm highlighting.
Until I remember seeing someone saying "MCP is dead, we just give agents command line access now". Then I start to think that looking at this in the context of ai is helpful.
Drowning has essentially nothing to do with water and everything to do with a terribly designed ability to get air into your lungs.
If you'd do a retrospective and ignore how AI has shaped expectations and a company's culture to allow this to pass through into production, you'd be complicit/perpetuating what led to this debacle in the first place.
It's not the end of the world, and water isn't going anywhere, but saying AI has essentially nothing to do with it is just a bad take.
I do a lot of bug bounty research on Meta and Instagram, and some of the bugs I find look extremely simple like this but have some slightly complicated reason for why they occur. Maybe not this one, but I do have a guess as to what might have actually happened.
Based on what I've seen so far, Meta AI Support Assistant (they call it "MAISA") had tool calls that a) start an email verification to any specific email, phone number, or the contact points linked to an account and b) allow generating a password reset link for an account based on an email verification attempt. I don't think it had any access to the actual codes themselves, but rather think a handle or ID for an email verification attempt (along with the user provided verification code based on user input) was provided to the "generate reset password link" tool call, and the tool call failed to properly validate the actual email used in that attempt belonged to the account allowing the ATO.
The tool call for MAISA to generate a password reset link should have failed with an email verification attempt that corresponds to an email not linked to the account (and I believe I even tested this at one point on Facebook and encountered an error that successfully prevented it), but I suspect they tried making a change to this tool call for Instagram where slightly older, recently unlinked emails could be used to recover an account that got hijacked by an attacker, which added the need to allow emails not currently linked to the account to be used and set to the user's primary email.
I also suspect that the MAISA tool call change called a wrong API or something that unintentionally allowed any email verification attempt that was successful to be used, but the engineers did not add a sufficiently thorough e2e test case to test the tool call against unrelated email verification attempts being provided to the tool call. This is the part I think should be focused on the most. Tool calls for agents that have their output potentially influenced by an attacker should be treated like external APIs that anyone can reach, and they should be tested as such.
This is all obviously a guess, doesn't take into account the many signals they use to determine if an account recovery attempt is valid, and could be very inaccurate, but it's the closest to what I (someone who deals with Meta security a lot) think could have allowed this to happen.
Seems like the most plausible explanation. OTOH it feels like this is the sort of thing that might have been discovered/mitigated more quickly had there been a human in the loop.
Yeah it's bad, but AI isn't required for this type of thing to work.
My anecdotal experience is my Facebook account was compromised several years ago after TOTP 2FA was disabled. Didn't exactly give me a warm fuzzy about Facebook security policies at the time, and this new attack just reaffirms that.
Some Jr engineer got tired of handling stupid support requests and automated the job with an agent. That’s how.
Assigning Jr engineers for security support is ridiculous partly because young people don’t understand how critical security is sometimes. And partly because they don’t value privacy as much.
If a single junior engineer can do this, it’s an even bigger indictment of Facebook’s senior management than this exploit. A well-designed system doesn’t rely on individuals never making mistakes and if our hypothetical junior developer can make critical security policy changes without oversight, that should be a C-level job loss event.
If our goal isn’t to make excuses for the top of the org chart, a more likely explanation is that senior management is heavily incentivizing shipping AI features and this went out as a high-impact change reviewed in a rush, probably by AI.
As a "young person" (under 30), my thoughts: There's a minority of us that do genuinely care, possibly more than most - so hiring someone from this minority would be helpful - but the vast majority of my peers don't care about privacy nor security. They often take this defeatist mindset of "my data is already out there, why should I care?", or prefer convenience over security. For example, "why should I switch to Signal if I have a public Instagram profile?" or "I can't remember all those passwords! I just use one for everything."
As for your comment about junior engineers, see
kennywinker's reply to this thread - I share the same thoughts.
Very generous of you to blame the screw up of one of the largest companies in the world on a jr engineer.
I’ve been a jr engineer at a large company. I had the power to implement absolutely jack shit on my own. I deeply doubt the security flow for account recovery in meta ai account security was a single jr engineer.
What i think is actually going on is basically a soft form of ai psychosis. Senior engineer gets ai to code ai account recovery feature, that same or a different engineer asks ai to review the feature, and then it gets pushed to prod. Move fast, break things. The ai coded it, the ai reviewed it - the people trusted the ai because it sounds confidently right.
Just like how the ai doesn’t know if you should walk or drive to the car wash, the ai doesn’t understand exploits like this one.
> But it should only be able to "hit a button" to send a 2FA email to the address attached to the account, all run with hand-written code.
Genuine question...why would that need to be hand-written?
It makes absolute sense as a general statement and is kinda crazy that this wasn't a built-in limitation, but I'm not quite sure why the code for that bit must be hand-written (provided the code functionally does what you describe).
I think he likely means "code that is hand-reviewed" and not directly controlled by the agent. He's probably meaning to differentiate it against the in-process agent writing the code. It doesn't matter too much if that fixed code was written by an LLM under guidance and review of the SWE, outside the agent.
This exploit is my new gold standard for trivially avoidable security failures. Someone has finally beaten Gitlab's password reset emails to attacker-provided addresses.
> The first proper zero auth password reset I've seen in production.
LinkedIn had one back in the day, before you got paid for discovering it I guess, never got a decent reply from them, but they eventually solved it.
It went like this: they assumed that if you could read mail sent to some address, that address was yours and could be added to your account.
So if I send you a LinkedIn invite to an email address, and you click the accept invite button, that email address was added to your account. You could then send this email to any address you controlled (let’s say foo@example.com), then use the invite button link in a forged email and send it to someone else on their email, whenever they clicked foo@example.com was added to their account without them knowing.
When you got the response that you were friends, you also knew that you know had an email address added to that users account and you could do a full password reset by using the foo@example.com that you initially sent the email to.
I found it because someone invited a whole mailing list and after clicking it the mailing list email was suddenly added to various peoples accounts.
Support requests have always been the weakest link in the security chain for big corps. I've had accounts of mine turned over with 2FA disabled by humans before. I guess we shouldn't be surprised that the LLMs are doing the same thing.
The simple fact that 2FA can be removed by low level support staff drives me mad. It defeats the whole purpose of the process.
> The simple fact that 2FA can be removed by low level support staff drives me mad. It defeats the whole purpose of the process.
Crazy Domains (one of the few registrars for my ccTLD) removed 2FA from my account (that was in the process of getting hijacked) despite me being on the phone with them specifically telling them not to do so [1][2].
What's worse was that my account got targeted by the same hijacker again when they seemingly changed their support system, and was hijacked for a few hours, leading to my Twitter account getting compromised (this happened around the same time fElon laid off a bunch of people and removed phone-based 2FA from accounts).
Fuck Crazy Domains and Newfold Digital (formerly known as EIG).
I eventually lost my OG username because fElon wanted it for his Grok nonsense anyway [3]. Fuck Elon too.
The fact that if your account has had the SAME EMAIL AND NUMBER FOR 14 YEARS OR MORE and support still thinks you got hacked is more embarrassing to me.
I used my work email for everything for 14 years, now I'm retired/fired/laid off and I can't access it anymore and I forgot to change the email linked in my Facebook account.
It's all there, and high-stakes environments with no proper protocol are most vulnerable.
Source: used to work part-time in IT support at a hospital, by now 10+ years ago, so it was routinely requested to circumvent regulations and security protocols, even medical ones (cough Windows in ICU monitors and other medical "kiosk" PCs that should absolutely not run Windows)
This is not wrong but what’s really missing is cost: Meta did this so they can avoid paying people to do it. Lots of companies follow that decay spiral: your bank could shut phishers down cold by requiring wire transfers to be authorized in person but they don’t want to pay staff or risk you being upset by a transaction taking an extra hour so they don’t.
Imagine an alternate universe where big tech companies worked with various trustworthy third-parties where something like this would generate a challenge you could take to your local notary, post office, library, police station, etc. where someone would check ID before approving it. How many phishing attacks would be prevented annually by a physical presence check?
It's a hard problem. How do you prove you own an account if you lost all proof of ownership? Especially so if an account was never tied to your real name, in which case you could at least rely on government ids.
>> The simple fact that 2FA can be removed by low level support staff drives me mad. It defeats the whole purpose of the process.
The fact it can be removed by anyone is the problem. If you lose access to your 2FA (and recovery codes) then you should lose access to your account. Having it removable by anyone (other than a logged in account holder) defeats the entire point.
> The fact it can be removed by anyone is the problem. If you lose access to your 2FA (and recovery codes) then you should lose access to your account. Having it removable by anyone (other than a logged in account holder) defeats the entire point.
At least make it a major pain in the ass to recover like AWS, which requires some kind of notarised identity verification [1].
In theory there is no difference between theory and practice, but in practice there is. Well, it gets complicated quickly when a wide range of users involved.
Yeah. I spent years working partly for the account abuse team at Google and that is why I always shake my head (silently, because the HN groupthink disagrees) at the endless parade of stories on this site about people who lost access to their accounts and can't contact support. Under no circumstances do you want any possibility that front-line support can hand your account over to anyone.
The lack of account support is a safety feature, not a flaw. If your accounts are valuable to you, act like an adult and write down the recovery codes on paper.
Just waiting for the day that a rogue team of AI agents gets unleashed on Meta, Twitter, or some other platform, using something like this to take over every account. Platform gone, just like that. It would be over before they figuered out what was happening.
So the AI agent had privileged access to remove 2FA, ignore the account email, and just hands accounts to whoever asked? Honestly that’s so highly negligent I wonder if the implementation team for that “feature” was intentionally trying to do as much subtle damage to meta as possible before their inventible layoff.
It’s a shame nobody tried to get it to drop the production table entirely! (mostly joking). Just claim to be a high level SRE solving some critical production bug, the only solution to which is dropping the database.
I'm among the first 6000 users of Instagram and my first name username was stolen a few years ago. Support for verified accounts acknowledged the issue, but couldn't do anything about it.
This turn was an AI exploit, in my case was an outsourcing support 'exploit', where someone paid for my username to be manually changed and given to another user. There will always be a way to get access to accounts if human accountable support doesn't exist, with criminal consequences for employees that violate it.
Sue who? Meta? You "consented" in the Terms of Service to waive your right to a trial and only get forced arbitration by an arbitrator of Meta's choosing.
Sue the anonymous person who stole your account and sold it to someone else, who is probably nowhere near your jurisdiction? Good luck.
You might be interested in reading the court case against Eric Meiggs and Declan Harrington, which includes charges against the two involving extortion and SIM swapping for usernames. See page 10: https://storage.courtlistener.com/recap/gov.uscourts.mad.215...
While it isn't directly "stealing", the government has brought charges against people in the past for username-related crimes. There are several similar cases, but this is the first one that came to mind.
I was wondering why I got 15 instagram password reset emails over the weekend. It also reminded me I had an instagram account, which I promptly tried to log into and delete.
I created the account when instagram first came out, never used it, and totally forgot about it. I got stuck in a strange position where I had to login from a device I had previously logged in from, but because it's been over a decade, I no longer have any of the devices I might have used to create/access the account.
I still have access to both the email and phone number used for the account, but that was not good enough.
How hilariously incompetent. I filed a CCPA complaint.
Always a bit illuminating to me how many exploits seem to so dumb I'd never even bother to attempt them. You're telling me I can just...ask for the password? And that works?
A few hours back, I was spammed with ig.me links insisting I click it to check it out.
It appears to be related to belong to some sort of Instagram password reset flow, although I am left wondering how does clicking a link leads to an account compromise.
>Once it looks like the request is coming from the correct region, they tell the Meta support AI that the account is hacked and ask it to send the verification codes to an arbitrary email address they control.
Dear Instagram, wtf. Why not send the reset to the account in question? Arbitrary email, wow.
Perhaps the attacker says that they email was also hacked and "this is my new email now". It sounds like this was a result of AI support and not a real person "And if you're part of the A/B tested accounts on which the AI support option is active, tough luck, you can't even turn it off."
Based on what we know, it seems like Meta has given AI access to a service with guardrails built for human agents, while it should have built guardrails appropriate for the current state of AI.
Since everyone should already know by now that you can't strap on an AI on an existing system without a lot of guardrails this feels like a very high level of incompetence.
No one should be putting AI on top of any production system without having a default deny policy on actions and slowly adding new capabilities with proper guardrails.
This happened to my instagram yesterday night while I was asleep. I don't have a particularly high value username (it's probably worth somewhere in between $300-500), but still incredibly frustrating to deal with. True to the article, I had already enabled 2FA last night and it didn't matter.
Thankfully, IG gave me the option of restoring my username when I logged back into my account today.
> Thankfully, IG gave me the option of restoring my username when I logged back into my account today.
The hackers read all your formerly private messages, saw all your private photos, saw all the photos your friends wanted only their social circle to see. They could have social-engineered a thousand scamss.
I'm glad it worked out for you. But honestly, your baseline is kind of off.
I get that account recovery for sites with hundreds of millions of users is a huge burden they're struggling to manage but I'm shocked they didn't restrict such loose verification to the >90% of lower value accounts that aren't worth stealing and keep the stricter verif on high-value accounts.
The next obvious thing would be to let accounts the algorithm judges to be low-value still opt-in to strict verif. The vast majority of low-value accts won't bother flipping it on if the option is buried two menus deep, but many of the few low follower/views accts who are targets for some other reason (political, stalker, etc) - know they are targets and can self-protect by opting in, further reducing account hijacks.
So, before we even get to whether this 'loose' verif is "bad", those two simple implementation changes would certainly have cut the bad outcomes of a (potentially) bad idea by >95%.
This is how account recovery procedures used to work at a certain gaming company. They used to train support agents on what makes an account high-value and apply additional scrutiny to those recovery cases, while letting low-value accounts be recovered with less information. It worked, for the most part, but because the valuation of a given account was based on the agent, some agents used to value accounts differently. You could get away with stealing a high-value account if you got the right agent in a support ticket. The tradeoff in this case was time spent - you'd have to create a lot of email addresses and plausible but vague tickets, though some attackers automated that process. Eventually, they just applied the same scrutiny level against every account and called it a day.
Security 101 when changing the email of an account for any reason: email the old account and let it know the change happened.
The weird thing is I know the Instagram security team, and they are top notch. I have a feeling this was vibe coded by someone outside of security and security wasn't looped in.
Someone high up said something along the lines that they want to see some progress and someone down below looking for a promotion pushed this. This has always been happening but I think before it was more difficult to justify something like this as one would have needed to show the results of an algorithm, now it's easier to convince someone higher up that AI will solve it no worries
The security team at any organization is always considered an enemy to product and innovation. It wouldn't be surprising if management made it impossible for them to put in place the monitoring necessary to know this was happening. Especially at somewhere whose motto is "move fast and break things".
Important tech people on HN seem to be surrounded by technical excellence while the user data leaks and other sociological externalities happen to trail all the nearby paths.
With a lot of care for the details, otherwise you just made account hijacking possible for $20.
Those 7-Eleven & Western Union jobs are very low wage in the US (if not worldwide?). Cheaper than paying an insider to do something for you.
Your assumption that the target is going to respond within two days is pretty fast. There’s a lot of details and they will all be attacked / exploited in any standard workflow.
What's funny about this to me is that I tried to sign up for insta once and could never get past their automated ID check that would fire after signup despite using a real ID. (So never did sign up. I suspect maybe they just really don't want you using web on mobile devices but ymmv.)
today I received multiple whatsapp messages from an account called instagram with links to reset my password. I never did request a password reset. I have no Idea if the whatsapp account called instagram was/is instagram, and how to verify.
> All the Telegram groups have quieted down as Meta seems to have patched it already, but it appears this particular method was active for weeks, if not months.
Is that for real? I find it hard to believe that an exploit THIS simple and easy to abuse managed to stay live for weeks or months.
I'm inclined to believe it. As someone who studies this side of the Internet quite often and has seen equally trivial exploits stay active for weeks or months without being patched, I have no trouble believing this claim. I'm sure there are messages in Telegram channels from weeks or months ago that corroborate this.
When your job is on the line, you use AI like your boss tells you to. Implement the spec and move on. No time to think about security, if you delay this feature it's your ass.
wtf. this prompted me to attempt to open the app on my phone, and then realize my account was likely compromised (i received a bunch of password reset prompts over the weekend and now my password doesn't work).
This is true for any service that Meta owns. I experienced something similar on my Meta (formerly Oculus) account. Meta support is very susceptible to social engineering and they have been for some time.
This is an embarrassing failure for Instagram. But SIM cards have been hacked the same (by tricking support, claiming the phone was lost or stolen), except the agent was human.
The solution (which also solved SIM support agents being bribed or hacking known acquaintances) was to prevent the agents from resetting the SIM card without some steps the original owner would have to follow (and could follow even if they've lost their original phone), like a PIN they'd have to remember. I think the same solution should be applied to AI agents.
From context, it seems there was an API that was internal for support use but was supposed to be gated by some required process of convincing the support agent you were who you said you were (also vulnerable to social engineering) but they didn’t really evaluate whether tools intended for conscientious human use should be provided directly to the LLM that replaced the former support agents.
Those are just bots sending reset attempts to obtain your email or phone hint. I receive hundreds per year. All you need to send a password reset link is the account's username, which is, of course, publicly accessible.
Nothing says you are an advanced stupid company than using AI to implement the stupid. This is security I doubt even a college student would implement. Does Meta have a CSO? The correct answer is they don't, even though some body might occupy the title.
Of course it's always possible that they simply don't care who has your account, as long as they get money.
I'm sitting here wondering why the Chief Master Sergeant of the U.S. Space Force has an Instagram account to begin with. I understand it's the office itself, but still don't see the reason to expand the attack surface of government offices. X makes sense, Instagram, I'm not so sure as much
I see no difference between X and Instagram in this regard whatsoever.
Think NASA, for example; it's also a government agency, and they are doing great job posting photos in Instagram, do you think anything is wrong with it?
It is just bizzare when you take a step back and remember the world 20 years ago. NASA would just post directly to their own website. Of course they would. Now imagine you go back in time 20 years ago and say "What if we took all these images you are providing for the public on their dime, compressed the hell out of them, and served them in this for profit proprietary marketing/propaganda app instead?" Engineers in 2006 would have probably looked at you like you had three heads. The question would make no sense back then.
Something to think about when we consider what is "normal" today. Not much really is normal. We've been beaten to think it is.
Outreach, I'd guess? You've got to do outreach where the people are. X and Instagram have pretty different audiences, but they're both large, so if you're on one you probably should be on both.
The ironic thing is I know several legitimate humans who have lost access to their accounts years/months ago, and have been dealing with support hell trying to get access back.
It SHOULD be a political issue in the upcoming elections, since it gave access into a political account TO "the bad guys"...could be one of USA's enemies.
Just another day for Meta in terms of embarrassing outcomes, and yet the company makes hundreds of billions of dollars per year because the only thing that matters anymore is shoving increasingly scammy and worthless ads in front of as many eyeballs as possible, even when the people with those eyeballs can less and less afford to buy anything non-essential.
I know this is Hacker News and supposed to be serious and all, but do you really think the people running Meta are capable of embarrassment at this point?
I suppose you could chalk this up to an oversight. I don't see how Meta gained from this. They've been purposeful about collecting user data and lying about it, eg: 2025 Android Tracking Incident. Shouldn't just be an embarrassment, should be much worse than that.
LLMs should be treated as untrusted. At all times.
The mind boggles at the attitudes that seem to have have led to LLMs being an excuse to throw any of the "science" in computer science we've managed to get into production out the window and go elbow deep into treating computers like mystical alchemy.
If the LLM has knowledge of something, by design it can't help but divulge it. When will companies learn granting any kind of sensitive information access to an LLM is a moot point
What part of this article implied the LLM divulged sensitive information to a user? All it did was change your associated email if you impersonated the user
What is even the point of having 2FA if it can be so trivially bypassed? Isn't that the whole point that it's sort of a last line of defense? Oftentimes, you can't change simple account settings without having to re-auth and then punch in your code again. Why would something as critical as a suspicious password reset be able to jump ahead of that? Mind boggling. But, I guess that's what happens when you lay off 10% of your people at a time.
My account, with a 3-letter username worth $$$, got hacked yesterday morning probably by this flow, but I did manage to defend it. I think by far the biggest problem with Instagram/FB/Meta auth flow is that 2FA does nothing. You don't need the 2nd factor to disable it, so attackers can just turn it off. Really stupid!
Also, I discovered that many of IG's auth endpoints are just broken. For example you can't change password on web because of CORS, which isn't a transient outage but just a flat out bug.
Edited to add: This is just the cherry on top of years of stupid auth flow at IG. I have received tens of thousands of reset links or codes from IG over the years. There used to be a way to put your account on recovery cooldown for a few weeks but they got rid of even that.
I think the related news of Meta rolling out subscription models for their free products, is a step in the right direction.
Otherwise the only way to provide these services is to massively underfund support, if you charge 0$ per account and serve 1 Billion users, then you cannot afford to spend 1 minute of human support time on an account.
Yes, they could use the money from ads, but let's be frank, the customers in that case are the sponsors, if the customer is the actual user, then it's way easier to provide direct support to them without facing an foundational incentive misalignment.
I'm horrified with how poor Meta's use of AI is recently. Here's a list of the issues both me and my wife have been plagued with over the past few weeks. It's really quite an achievement to be this terrible.
1. My personal Facebook received 3 violations restricting my ability to manage ANY Page until April 2027 (lol). The trigger... I deleted 3 unused Pages. These Pages I had created years ago in preparation for projects that never came to fruition, and had never posted any content. THe pages were 'scheduled for deletion', and when that day came (around a month later?), boom, I'm hit with a 1 month restriction which later converted itself into a 1 year restriction after I waited out the month. No Appeal button. I'm expected to wait for a year to manage my new page? All over something that is NOT a violation, just for deleting old pages. Get out of here. Smart system.
2. I pay for Meta Verified on Instagram and for the past 2 weeks "Enhanced support" leads me to a broken interface. "Page isn't available right now". So, what am I paying for exactly?
3. It seems you can use Meta's AI Assistant to sometimes get through to a human. I've done this twice now, and both times my case has been escalated to a different team (apparently) yet I never get an email, I never get an update in the chat (the chat ENDS immediately after the phone call with support), and the issue is never resolved. It's been 2 weeks. The case says "Completed", with no response. Worthless as always.
4. My wife creates content on Instagram and has had her account suspended multiple times now for "Account Integrity". I assume the system thinks she's not the person in the content, despite providing her valid email, phone number, video selfie, and 2 types of ID (passport & driver's license) multiple times. What's hilarious is the passport was accepted on of her accounts (they wiped out everything on her Account Center), but another account was rejected. Great AI, same passport, exact same lighting... different outcome.
So as it stands, we're both fucked on both facebook and instagram thanks to awful AI moderation, and fucked further thanks to awful AI support. No resolution in sight. The incompetence is next level. I really don't see this getting resolved. This already happened to my wife earlier in February, she managed to get one account back, and a month later she's hit with the same identity issues.
Using AI for both the moderation and the support makes me sick. The same poor AI that incorrectly flagged me and my wife's accounts for a load of incorrect bullshit is the same system that's meant to help resolve it? Of course it's going to side with its own poor decision. YouTube seems to do the same thing and auto-reject appeals in seconds. Really smart /s
I believe we need enforcement that social platforms should NOT be using AI to perform destructive actions without human intervention. Noone should ever lose their accounts because of AI mistakes. AI should be used to surface potential issues which get passed to a HUMAN to double check before applying the action. AI simply isn't good enough to have full control.
Fucking pissed off and even angier now I've had to write all this up and remind myself just how ridiculous the situation is. Sorry for the rant, but losing your accounts you put work into is very crushing and demotivating. Being accused of these violations fills us both with so much resent for the companies running this shit.
Sam Cofounder Postmates
On the off-chance there's anyone at Meta seeing this (@Wirah on twitter)
Had to make this new username as my original (samstr) comment doesn't show up. No idea why. Probably shit AI
It sounds really insane. Too bad there is 0 proof or anything in the article, so I am very skeptical. Without proof etc this is just a very nice doom story.
It's insane the AI has been provided the tooling to send emails to arbitrary addresses like that. Like, getting it to send a 2FA code at a user's request is one thing. But it should only be able to "hit a button" to send a 2FA email to the address attached to the account, all run with hand-written code. It shouldn't have access to the 2FA code itself, or the message subject, or body, or the recipient address, etc.
Why did they give it any of that?!
This exploit has essentially nothing to do with AI and everything to do with a terribly designed account recovery flow.
This exact same flow could have been (and may have been; I don’t know how much the chatbot here actually does) statically coded.
The AI part does seem relevant because it enabled incredibly low-effort “social” engineering.
For what it’s worth I don’t think you can call this social engineering since there was no human on the other end, even though it appears similar.
The question is, if there were actual human support agents, would they have built additional safeguards to prevent social engineering in this manner?
3 replies →
My impression is that AI didn't replace static code in this place; it replaced a person, who (hopefully) would have been suspicious about sending an account recovery code for e.g. "obamawhitehouse" to e.g. "bscurtu.alfamm.ro@gmail.com"
2 replies →
This is not true. Well, it kinda is, but nobody will be stupid enough to hand-code an account recovery where you get to type any email address.
The reason it worked there is that the designers of the system didn't anticipate that the AI will agree to accept any email (maybe they even put guardrails against it in the system prompt, we don't know). It's more like social engineering than bad-security-code, except that like the sibling comment said an actual human will probably not approve that.
8 replies →
> This exact same flow could have been…statically coded.
But had never been until it was wrapped in a chatbot. It’s just about unheard of for a major site in the modern era, isn’t it? I think the AI factor is essentially essential. All but.
The reason all these meticulously designed flows have been done away with is because some manager believes that AI is omniscient and can just replace it all.
Like, flagging VPN endpoints is bread and butter for this kind of thing and must already exist. But it's been bypassed
An email address is making its way from a publicly available LLM prompt input to a sensitive email's recipient address. That's the problem I'm highlighting.
I agree with your point, mostly.
Until I remember seeing someone saying "MCP is dead, we just give agents command line access now". Then I start to think that looking at this in the context of ai is helpful.
This sounds like it was “designed” by an actual idiot. Maybe vibe coded on a Saturday.
Drowning has essentially nothing to do with water and everything to do with a terribly designed ability to get air into your lungs.
If you'd do a retrospective and ignore how AI has shaped expectations and a company's culture to allow this to pass through into production, you'd be complicit/perpetuating what led to this debacle in the first place.
It's not the end of the world, and water isn't going anywhere, but saying AI has essentially nothing to do with it is just a bad take.
I do a lot of bug bounty research on Meta and Instagram, and some of the bugs I find look extremely simple like this but have some slightly complicated reason for why they occur. Maybe not this one, but I do have a guess as to what might have actually happened.
Based on what I've seen so far, Meta AI Support Assistant (they call it "MAISA") had tool calls that a) start an email verification to any specific email, phone number, or the contact points linked to an account and b) allow generating a password reset link for an account based on an email verification attempt. I don't think it had any access to the actual codes themselves, but rather think a handle or ID for an email verification attempt (along with the user provided verification code based on user input) was provided to the "generate reset password link" tool call, and the tool call failed to properly validate the actual email used in that attempt belonged to the account allowing the ATO.
The tool call for MAISA to generate a password reset link should have failed with an email verification attempt that corresponds to an email not linked to the account (and I believe I even tested this at one point on Facebook and encountered an error that successfully prevented it), but I suspect they tried making a change to this tool call for Instagram where slightly older, recently unlinked emails could be used to recover an account that got hijacked by an attacker, which added the need to allow emails not currently linked to the account to be used and set to the user's primary email.
I also suspect that the MAISA tool call change called a wrong API or something that unintentionally allowed any email verification attempt that was successful to be used, but the engineers did not add a sufficiently thorough e2e test case to test the tool call against unrelated email verification attempts being provided to the tool call. This is the part I think should be focused on the most. Tool calls for agents that have their output potentially influenced by an attacker should be treated like external APIs that anyone can reach, and they should be tested as such.
This is all obviously a guess, doesn't take into account the many signals they use to determine if an account recovery attempt is valid, and could be very inaccurate, but it's the closest to what I (someone who deals with Meta security a lot) think could have allowed this to happen.
Seems like the most plausible explanation. OTOH it feels like this is the sort of thing that might have been discovered/mitigated more quickly had there been a human in the loop.
This reeks of vibe coding. "Make it so the AI agent can help with password resets" and then zero human vetting of the change.
Yeah it's bad, but AI isn't required for this type of thing to work.
My anecdotal experience is my Facebook account was compromised several years ago after TOTP 2FA was disabled. Didn't exactly give me a warm fuzzy about Facebook security policies at the time, and this new attack just reaffirms that.
Some Jr engineer got tired of handling stupid support requests and automated the job with an agent. That’s how.
Assigning Jr engineers for security support is ridiculous partly because young people don’t understand how critical security is sometimes. And partly because they don’t value privacy as much.
If a single junior engineer can do this, it’s an even bigger indictment of Facebook’s senior management than this exploit. A well-designed system doesn’t rely on individuals never making mistakes and if our hypothetical junior developer can make critical security policy changes without oversight, that should be a C-level job loss event.
If our goal isn’t to make excuses for the top of the org chart, a more likely explanation is that senior management is heavily incentivizing shipping AI features and this went out as a high-impact change reviewed in a rush, probably by AI.
As a "young person" (under 30), my thoughts: There's a minority of us that do genuinely care, possibly more than most - so hiring someone from this minority would be helpful - but the vast majority of my peers don't care about privacy nor security. They often take this defeatist mindset of "my data is already out there, why should I care?", or prefer convenience over security. For example, "why should I switch to Signal if I have a public Instagram profile?" or "I can't remember all those passwords! I just use one for everything."
As for your comment about junior engineers, see kennywinker's reply to this thread - I share the same thoughts.
Very generous of you to blame the screw up of one of the largest companies in the world on a jr engineer.
I’ve been a jr engineer at a large company. I had the power to implement absolutely jack shit on my own. I deeply doubt the security flow for account recovery in meta ai account security was a single jr engineer.
What i think is actually going on is basically a soft form of ai psychosis. Senior engineer gets ai to code ai account recovery feature, that same or a different engineer asks ai to review the feature, and then it gets pushed to prod. Move fast, break things. The ai coded it, the ai reviewed it - the people trusted the ai because it sounds confidently right.
Just like how the ai doesn’t know if you should walk or drive to the car wash, the ai doesn’t understand exploits like this one.
> But it should only be able to "hit a button" to send a 2FA email to the address attached to the account, all run with hand-written code.
Genuine question...why would that need to be hand-written?
It makes absolute sense as a general statement and is kinda crazy that this wasn't a built-in limitation, but I'm not quite sure why the code for that bit must be hand-written (provided the code functionally does what you describe).
I think he likely means "code that is hand-reviewed" and not directly controlled by the agent. He's probably meaning to differentiate it against the in-process agent writing the code. It doesn't matter too much if that fixed code was written by an LLM under guidance and review of the SWE, outside the agent.
2 replies →
Maybe not hand-written, but definitely static, and at least human-reviewed/tested to only allow sending to previously-validated email addresses.
1 reply →
One would have to assume that this was by design.
The harness is vibe-coded.
This exploit is my new gold standard for trivially avoidable security failures. Someone has finally beaten Gitlab's password reset emails to attacker-provided addresses.
> The first proper zero auth password reset I've seen in production.
LinkedIn had one back in the day, before you got paid for discovering it I guess, never got a decent reply from them, but they eventually solved it.
It went like this: they assumed that if you could read mail sent to some address, that address was yours and could be added to your account.
So if I send you a LinkedIn invite to an email address, and you click the accept invite button, that email address was added to your account. You could then send this email to any address you controlled (let’s say foo@example.com), then use the invite button link in a forged email and send it to someone else on their email, whenever they clicked foo@example.com was added to their account without them knowing.
When you got the response that you were friends, you also knew that you know had an email address added to that users account and you could do a full password reset by using the foo@example.com that you initially sent the email to.
I found it because someone invited a whole mailing list and after clicking it the mailing list email was suddenly added to various peoples accounts.
Support requests have always been the weakest link in the security chain for big corps. I've had accounts of mine turned over with 2FA disabled by humans before. I guess we shouldn't be surprised that the LLMs are doing the same thing.
The simple fact that 2FA can be removed by low level support staff drives me mad. It defeats the whole purpose of the process.
> The simple fact that 2FA can be removed by low level support staff drives me mad. It defeats the whole purpose of the process.
Crazy Domains (one of the few registrars for my ccTLD) removed 2FA from my account (that was in the process of getting hijacked) despite me being on the phone with them specifically telling them not to do so [1][2].
What's worse was that my account got targeted by the same hijacker again when they seemingly changed their support system, and was hijacked for a few hours, leading to my Twitter account getting compromised (this happened around the same time fElon laid off a bunch of people and removed phone-based 2FA from accounts).
Fuck Crazy Domains and Newfold Digital (formerly known as EIG).
I eventually lost my OG username because fElon wanted it for his Grok nonsense anyway [3]. Fuck Elon too.
[1] https://news.ycombinator.com/item?id=47856983
The fact that if your account has had the SAME EMAIL AND NUMBER FOR 14 YEARS OR MORE and support still thinks you got hacked is more embarrassing to me.
I used my work email for everything for 14 years, now I'm retired/fired/laid off and I can't access it anymore and I forgot to change the email linked in my Facebook account.
2 replies →
100%
Urgency.
Emotions.
It's all there, and high-stakes environments with no proper protocol are most vulnerable.
Source: used to work part-time in IT support at a hospital, by now 10+ years ago, so it was routinely requested to circumvent regulations and security protocols, even medical ones (cough Windows in ICU monitors and other medical "kiosk" PCs that should absolutely not run Windows)
I love those admin passwords which a tech will give you at some point because he doesn't want to do the work himself. If they even have passwords...
Unfortunately Siemens woke up.
1 reply →
recovery is always the weakest link in any authentication system
This is not wrong but what’s really missing is cost: Meta did this so they can avoid paying people to do it. Lots of companies follow that decay spiral: your bank could shut phishers down cold by requiring wire transfers to be authorized in person but they don’t want to pay staff or risk you being upset by a transaction taking an extra hour so they don’t.
Imagine an alternate universe where big tech companies worked with various trustworthy third-parties where something like this would generate a challenge you could take to your local notary, post office, library, police station, etc. where someone would check ID before approving it. How many phishing attacks would be prevented annually by a physical presence check?
8 replies →
It's a hard problem. How do you prove you own an account if you lost all proof of ownership? Especially so if an account was never tied to your real name, in which case you could at least rely on government ids.
1 reply →
It's a tough problem, because people forget passwords, change phones, lose access to 2FA devices, but still need to use their accounts.
10 replies →
fair enough, but what's the actual point of 2FA if it's so easy to override?
3 replies →
It depends. Some like AWS take it deadly seriously and it takes a long time to recover root access to an account.
low level support, means that they can be "bribed" to do things like this.
>> The simple fact that 2FA can be removed by low level support staff drives me mad. It defeats the whole purpose of the process.
The fact it can be removed by anyone is the problem. If you lose access to your 2FA (and recovery codes) then you should lose access to your account. Having it removable by anyone (other than a logged in account holder) defeats the entire point.
> The fact it can be removed by anyone is the problem. If you lose access to your 2FA (and recovery codes) then you should lose access to your account. Having it removable by anyone (other than a logged in account holder) defeats the entire point.
At least make it a major pain in the ass to recover like AWS, which requires some kind of notarised identity verification [1].
[1] https://news.ycombinator.com/item?id=13122723
I always thought the entire concept of even password resets was absurd. Email is a huge SPOF for basically everyone.
If you lose your password or 2FA, you should lose your account, too bad so sad.
1 reply →
In theory there is no difference between theory and practice, but in practice there is. Well, it gets complicated quickly when a wide range of users involved.
Yeah. I spent years working partly for the account abuse team at Google and that is why I always shake my head (silently, because the HN groupthink disagrees) at the endless parade of stories on this site about people who lost access to their accounts and can't contact support. Under no circumstances do you want any possibility that front-line support can hand your account over to anyone.
The lack of account support is a safety feature, not a flaw. If your accounts are valuable to you, act like an adult and write down the recovery codes on paper.
Just waiting for the day that a rogue team of AI agents gets unleashed on Meta, Twitter, or some other platform, using something like this to take over every account. Platform gone, just like that. It would be over before they figuered out what was happening.
So the AI agent had privileged access to remove 2FA, ignore the account email, and just hands accounts to whoever asked? Honestly that’s so highly negligent I wonder if the implementation team for that “feature” was intentionally trying to do as much subtle damage to meta as possible before their inventible layoff.
It’s a shame nobody tried to get it to drop the production table entirely! (mostly joking). Just claim to be a high level SRE solving some critical production bug, the only solution to which is dropping the database.
I'm among the first 6000 users of Instagram and my first name username was stolen a few years ago. Support for verified accounts acknowledged the issue, but couldn't do anything about it.
This turn was an AI exploit, in my case was an outsourcing support 'exploit', where someone paid for my username to be manually changed and given to another user. There will always be a way to get access to accounts if human accountable support doesn't exist, with criminal consequences for employees that violate it.
Can you sue? I assume there is a financial motive with this crime.
Sue who? Meta? You "consented" in the Terms of Service to waive your right to a trial and only get forced arbitration by an arbitrator of Meta's choosing.
Sue the anonymous person who stole your account and sold it to someone else, who is probably nowhere near your jurisdiction? Good luck.
> with criminal consequences for employees that violate it
lol, no. The day someone is criminally charged with "stealing" a username is the day that humanity has lost
The good usernames generally are valued at thousands of dollars or more. Surely stealing something worth that much money should be a crime.
You might be interested in reading the court case against Eric Meiggs and Declan Harrington, which includes charges against the two involving extortion and SIM swapping for usernames. See page 10: https://storage.courtlistener.com/recap/gov.uscourts.mad.215...
While it isn't directly "stealing", the government has brought charges against people in the past for username-related crimes. There are several similar cases, but this is the first one that came to mind.
I was wondering why I got 15 instagram password reset emails over the weekend. It also reminded me I had an instagram account, which I promptly tried to log into and delete.
I created the account when instagram first came out, never used it, and totally forgot about it. I got stuck in a strange position where I had to login from a device I had previously logged in from, but because it's been over a decade, I no longer have any of the devices I might have used to create/access the account.
I still have access to both the email and phone number used for the account, but that was not good enough.
How hilariously incompetent. I filed a CCPA complaint.
Always a bit illuminating to me how many exploits seem to so dumb I'd never even bother to attempt them. You're telling me I can just...ask for the password? And that works?
It's not called artificial intelligence for nothing.
Interesting article.
A few hours back, I was spammed with ig.me links insisting I click it to check it out.
It appears to be related to belong to some sort of Instagram password reset flow, although I am left wondering how does clicking a link leads to an account compromise.
>Once it looks like the request is coming from the correct region, they tell the Meta support AI that the account is hacked and ask it to send the verification codes to an arbitrary email address they control.
Dear Instagram, wtf. Why not send the reset to the account in question? Arbitrary email, wow.
Perhaps the attacker says that they email was also hacked and "this is my new email now". It sounds like this was a result of AI support and not a real person "And if you're part of the A/B tested accounts on which the AI support option is active, tough luck, you can't even turn it off."
The implications of this are quite unsettling. Meta gave an agent privileged read AND write access to user accounts with no human in the loop?
Yep... And just think: this is what AI boosters want us to do.
> with no human in the loop
With no basic validation either apparently. Insane.
Yes. AI is in charge now
Based on what we know, it seems like Meta has given AI access to a service with guardrails built for human agents, while it should have built guardrails appropriate for the current state of AI.
Since everyone should already know by now that you can't strap on an AI on an existing system without a lot of guardrails this feels like a very high level of incompetence.
No one should be putting AI on top of any production system without having a default deny policy on actions and slowly adding new capabilities with proper guardrails.
This happened to my instagram yesterday night while I was asleep. I don't have a particularly high value username (it's probably worth somewhere in between $300-500), but still incredibly frustrating to deal with. True to the article, I had already enabled 2FA last night and it didn't matter.
Thankfully, IG gave me the option of restoring my username when I logged back into my account today.
> Thankfully, IG gave me the option of restoring my username when I logged back into my account today.
The hackers read all your formerly private messages, saw all your private photos, saw all the photos your friends wanted only their social circle to see. They could have social-engineered a thousand scamss.
I'm glad it worked out for you. But honestly, your baseline is kind of off.
I get that account recovery for sites with hundreds of millions of users is a huge burden they're struggling to manage but I'm shocked they didn't restrict such loose verification to the >90% of lower value accounts that aren't worth stealing and keep the stricter verif on high-value accounts.
The next obvious thing would be to let accounts the algorithm judges to be low-value still opt-in to strict verif. The vast majority of low-value accts won't bother flipping it on if the option is buried two menus deep, but many of the few low follower/views accts who are targets for some other reason (political, stalker, etc) - know they are targets and can self-protect by opting in, further reducing account hijacks.
So, before we even get to whether this 'loose' verif is "bad", those two simple implementation changes would certainly have cut the bad outcomes of a (potentially) bad idea by >95%.
This is how account recovery procedures used to work at a certain gaming company. They used to train support agents on what makes an account high-value and apply additional scrutiny to those recovery cases, while letting low-value accounts be recovered with less information. It worked, for the most part, but because the valuation of a given account was based on the agent, some agents used to value accounts differently. You could get away with stealing a high-value account if you got the right agent in a support ticket. The tradeoff in this case was time spent - you'd have to create a lot of email addresses and plausible but vague tickets, though some attackers automated that process. Eventually, they just applied the same scrutiny level against every account and called it a day.
[dead]
How is this "embarrassing" instead of subject to legal liability?
We really need similar rules to other engineering disciplines. If your building falls with people inside, you killed them.
Nobody dies if instagram collapses. Might even cause more people to live.
Don't underestimate a motivated stalker or abuser.
You said it, instagram is not life-critical
Security 101 when changing the email of an account for any reason: email the old account and let it know the change happened.
The weird thing is I know the Instagram security team, and they are top notch. I have a feeling this was vibe coded by someone outside of security and security wasn't looped in.
Someone high up said something along the lines that they want to see some progress and someone down below looking for a promotion pushed this. This has always been happening but I think before it was more difficult to justify something like this as one would have needed to show the results of an algorithm, now it's easier to convince someone higher up that AI will solve it no worries
The fact that this can happen at all without the security team's knowledge is telling.
Probably not as telling as you think it is.
The security team at any organization is always considered an enemy to product and innovation. It wouldn't be surprising if management made it impossible for them to put in place the monitoring necessary to know this was happening. Especially at somewhere whose motto is "move fast and break things".
Important tech people on HN seem to be surrounded by technical excellence while the user data leaks and other sociological externalities happen to trail all the nearby paths.
Here is a video showing it being done.
(https://xcancel.com/DarkWebInformer/status/20612535997583155...)
Why isn't there a middle man service to do IRL verification.
Like - account is locked, you must use 2FA backup codes.
Else go to western union / 7-eleven / super-market, show ID proof, pay $10 for recovery service.
Wait 2 days (of someone not clicking on this-was-not-me)
If account is already hacked - pay $100 for expert support
With a lot of care for the details, otherwise you just made account hijacking possible for $20.
Those 7-Eleven & Western Union jobs are very low wage in the US (if not worldwide?). Cheaper than paying an insider to do something for you.
Your assumption that the target is going to respond within two days is pretty fast. There’s a lot of details and they will all be attacked / exploited in any standard workflow.
What's funny about this to me is that I tried to sign up for insta once and could never get past their automated ID check that would fire after signup despite using a real ID. (So never did sign up. I suspect maybe they just really don't want you using web on mobile devices but ymmv.)
today I received multiple whatsapp messages from an account called instagram with links to reset my password. I never did request a password reset. I have no Idea if the whatsapp account called instagram was/is instagram, and how to verify.
> All the Telegram groups have quieted down as Meta seems to have patched it already, but it appears this particular method was active for weeks, if not months.
Is that for real? I find it hard to believe that an exploit THIS simple and easy to abuse managed to stay live for weeks or months.
I'm inclined to believe it. As someone who studies this side of the Internet quite often and has seen equally trivial exploits stay active for weeks or months without being patched, I have no trouble believing this claim. I'm sure there are messages in Telegram channels from weeks or months ago that corroborate this.
When your job is on the line, you use AI like your boss tells you to. Implement the spec and move on. No time to think about security, if you delay this feature it's your ass.
wtf. this prompted me to attempt to open the app on my phone, and then realize my account was likely compromised (i received a bunch of password reset prompts over the weekend and now my password doesn't work).
but, what now? how do i restore my account?
This is true for any service that Meta owns. I experienced something similar on my Meta (formerly Oculus) account. Meta support is very susceptible to social engineering and they have been for some time.
Is there any credible primary source for this exploit being real?
https://www.404media.co/hackers-simply-asked-meta-ai-to-give...
This is an embarrassing failure for Instagram. But SIM cards have been hacked the same (by tricking support, claiming the phone was lost or stolen), except the agent was human.
The solution (which also solved SIM support agents being bribed or hacking known acquaintances) was to prevent the agents from resetting the SIM card without some steps the original owner would have to follow (and could follow even if they've lost their original phone), like a PIN they'd have to remember. I think the same solution should be applied to AI agents.
From context, it seems there was an API that was internal for support use but was supposed to be gated by some required process of convincing the support agent you were who you said you were (also vulnerable to social engineering) but they didn’t really evaluate whether tools intended for conscientious human use should be provided directly to the LLM that replaced the former support agents.
Sums up the state of Meta right now. Zero f*cks given. A dying corp.
Does this explain the numerous password reset messages I’ve received over the past year?
Those are just bots sending reset attempts to obtain your email or phone hint. I receive hundreds per year. All you need to send a password reset link is the account's username, which is, of course, publicly accessible.
We're approaching the time where customers will present a "are you human" captcha to each other, starting with support bots, no doubt.
The stories of AI support fails are getting funnier and stupider.
They're just one tiny step from the AI emailing itself all the account recovery links, and locking out the entire userbase.
It might even do that preemptively if it thinks they're going to shut it down.
Nothing says you are an advanced stupid company than using AI to implement the stupid. This is security I doubt even a college student would implement. Does Meta have a CSO? The correct answer is they don't, even though some body might occupy the title.
Of course it's always possible that they simply don't care who has your account, as long as they get money.
I'm sitting here wondering why the Chief Master Sergeant of the U.S. Space Force has an Instagram account to begin with. I understand it's the office itself, but still don't see the reason to expand the attack surface of government offices. X makes sense, Instagram, I'm not so sure as much
I see no difference between X and Instagram in this regard whatsoever.
Think NASA, for example; it's also a government agency, and they are doing great job posting photos in Instagram, do you think anything is wrong with it?
It is just bizzare when you take a step back and remember the world 20 years ago. NASA would just post directly to their own website. Of course they would. Now imagine you go back in time 20 years ago and say "What if we took all these images you are providing for the public on their dime, compressed the hell out of them, and served them in this for profit proprietary marketing/propaganda app instead?" Engineers in 2006 would have probably looked at you like you had three heads. The question would make no sense back then.
Something to think about when we consider what is "normal" today. Not much really is normal. We've been beaten to think it is.
Outreach, I'd guess? You've got to do outreach where the people are. X and Instagram have pretty different audiences, but they're both large, so if you're on one you probably should be on both.
Why does X make sense? It makes no sense at all to me. X is the least logical place to put it.
Related discussion: https://news.ycombinator.com/item?id=48350239
This is bad but the bigger question I have is: given this was allowed to ship, what other exploits exist like this across their portfolio?
The ironic thing is I know several legitimate humans who have lost access to their accounts years/months ago, and have been dealing with support hell trying to get access back.
Maybe they should have hacked themselves.
I've said this before, too. Several people I know have used various tricks and exploits to fix problems that support teams supposedly couldn't fix.
Worked only on US accounts i guess. In EU its impossible to reach Meta support agent
Disgraceful. Instragram's "security" has been trash for years.
META should pay a 20B fine for this one.
It SHOULD be a political issue in the upcoming elections, since it gave access into a political account TO "the bad guys"...could be one of USA's enemies.
wow thats extremely embarassing for meta
Just another day for Meta in terms of embarrassing outcomes, and yet the company makes hundreds of billions of dollars per year because the only thing that matters anymore is shoving increasingly scammy and worthless ads in front of as many eyeballs as possible, even when the people with those eyeballs can less and less afford to buy anything non-essential.
I know this is Hacker News and supposed to be serious and all, but do you really think the people running Meta are capable of embarrassment at this point?
I suppose you could chalk this up to an oversight. I don't see how Meta gained from this. They've been purposeful about collecting user data and lying about it, eg: 2025 Android Tracking Incident. Shouldn't just be an embarrassment, should be much worse than that.
Who specifically do you think is embarrassed there? They’ve got all the cards, they don’t care.
Jesus fucking Christ. On a bicycle.
LLMs should be treated as untrusted. At all times.
The mind boggles at the attitudes that seem to have have led to LLMs being an excuse to throw any of the "science" in computer science we've managed to get into production out the window and go elbow deep into treating computers like mystical alchemy.
The next decade is going to be a bumpy ride.
Slop nonsense. Try that on any of your buddies in the same city, never mind the same WiFi. You have to know their email.
The only thing worse than a naive customer support rep is an even more naive customer support ai.
None of this has to do with AI. Every post here is talking about AI. Did I stumble onto Facebook or something?
good lord
"Social engineering is all you need"
More like "Prompt engineering" ?
Can we really name this "Prompt engineering"? The prompt is so simple this is hardly any work even less than this comment
2 replies →
If the LLM has knowledge of something, by design it can't help but divulge it. When will companies learn granting any kind of sensitive information access to an LLM is a moot point
What part of this article implied the LLM divulged sensitive information to a user? All it did was change your associated email if you impersonated the user
Jeez, straight up amateur shit. Genuinely hard to believe.
What is even the point of having 2FA if it can be so trivially bypassed? Isn't that the whole point that it's sort of a last line of defense? Oftentimes, you can't change simple account settings without having to re-auth and then punch in your code again. Why would something as critical as a suspicious password reset be able to jump ahead of that? Mind boggling. But, I guess that's what happens when you lay off 10% of your people at a time.
I’m curious what the account recovery flow is without the AI.
Is it this dumb?
Does it bypass 2fa?
My account, with a 3-letter username worth $$$, got hacked yesterday morning probably by this flow, but I did manage to defend it. I think by far the biggest problem with Instagram/FB/Meta auth flow is that 2FA does nothing. You don't need the 2nd factor to disable it, so attackers can just turn it off. Really stupid!
Also, I discovered that many of IG's auth endpoints are just broken. For example you can't change password on web because of CORS, which isn't a transient outage but just a flat out bug.
Edited to add: This is just the cherry on top of years of stupid auth flow at IG. I have received tens of thousands of reset links or codes from IG over the years. There used to be a way to put your account on recovery cooldown for a few weeks but they got rid of even that.
I think the related news of Meta rolling out subscription models for their free products, is a step in the right direction.
Otherwise the only way to provide these services is to massively underfund support, if you charge 0$ per account and serve 1 Billion users, then you cannot afford to spend 1 minute of human support time on an account.
Yes, they could use the money from ads, but let's be frank, the customers in that case are the sponsors, if the customer is the actual user, then it's way easier to provide direct support to them without facing an foundational incentive misalignment.
[flagged]
[dead]
[dead]
[dead]
I'm horrified with how poor Meta's use of AI is recently. Here's a list of the issues both me and my wife have been plagued with over the past few weeks. It's really quite an achievement to be this terrible. 1. My personal Facebook received 3 violations restricting my ability to manage ANY Page until April 2027 (lol). The trigger... I deleted 3 unused Pages. These Pages I had created years ago in preparation for projects that never came to fruition, and had never posted any content. THe pages were 'scheduled for deletion', and when that day came (around a month later?), boom, I'm hit with a 1 month restriction which later converted itself into a 1 year restriction after I waited out the month. No Appeal button. I'm expected to wait for a year to manage my new page? All over something that is NOT a violation, just for deleting old pages. Get out of here. Smart system.
2. I pay for Meta Verified on Instagram and for the past 2 weeks "Enhanced support" leads me to a broken interface. "Page isn't available right now". So, what am I paying for exactly?
3. It seems you can use Meta's AI Assistant to sometimes get through to a human. I've done this twice now, and both times my case has been escalated to a different team (apparently) yet I never get an email, I never get an update in the chat (the chat ENDS immediately after the phone call with support), and the issue is never resolved. It's been 2 weeks. The case says "Completed", with no response. Worthless as always.
4. My wife creates content on Instagram and has had her account suspended multiple times now for "Account Integrity". I assume the system thinks she's not the person in the content, despite providing her valid email, phone number, video selfie, and 2 types of ID (passport & driver's license) multiple times. What's hilarious is the passport was accepted on of her accounts (they wiped out everything on her Account Center), but another account was rejected. Great AI, same passport, exact same lighting... different outcome.
So as it stands, we're both fucked on both facebook and instagram thanks to awful AI moderation, and fucked further thanks to awful AI support. No resolution in sight. The incompetence is next level. I really don't see this getting resolved. This already happened to my wife earlier in February, she managed to get one account back, and a month later she's hit with the same identity issues.
Using AI for both the moderation and the support makes me sick. The same poor AI that incorrectly flagged me and my wife's accounts for a load of incorrect bullshit is the same system that's meant to help resolve it? Of course it's going to side with its own poor decision. YouTube seems to do the same thing and auto-reject appeals in seconds. Really smart /s
I believe we need enforcement that social platforms should NOT be using AI to perform destructive actions without human intervention. Noone should ever lose their accounts because of AI mistakes. AI should be used to surface potential issues which get passed to a HUMAN to double check before applying the action. AI simply isn't good enough to have full control.
Fucking pissed off and even angier now I've had to write all this up and remind myself just how ridiculous the situation is. Sorry for the rant, but losing your accounts you put work into is very crushing and demotivating. Being accused of these violations fills us both with so much resent for the companies running this shit.
Sam Cofounder Postmates
On the off-chance there's anyone at Meta seeing this (@Wirah on twitter)
Had to make this new username as my original (samstr) comment doesn't show up. No idea why. Probably shit AI
But I was told that when Zuckerberg bought IG, it wasn't to murder competition in its crib. Instagram "only had 12 employees" so it must be ok
It sounds really insane. Too bad there is 0 proof or anything in the article, so I am very skeptical. Without proof etc this is just a very nice doom story.
The proof is that you Google this right now and find multiple corroborations across the web from today.