> If you were a Mercor contractor and you believe your voice may already be in circulation, ORAVYS will analyze the first three suspect samples free of charge.
Awesome, if you're a victim of an AI company having your voice, you can help yourself by sending another AI company your voice!
> Audio is never used to train commercial models without explicit consent
I'm sure Mercor has explicit consent as well, legal teams are reasonably good at legally covering their asses with license terms.
The irony runs deeper than the free analysis offer. The whole Mercor contractor relationship was this exact pattern: hand over studio-quality voice recordings and ID scans to get paid for data labeling work that didn't require either. "Explicit consent" was buried in the terms, and people clicked through because they needed the paycheck.
Now 40k people have learned that biometrics aren't passwords. You can't rotate your voice.
> Now 40k people have learned that biometrics aren't passwords. You can't rotate your voice.
Voices aren't strong.
There just aren't that many unique characteristic parameters behind a voice - it's largely dictated by an evolutionary shared shared larynx and vocal tract. They aren't fingerprints.
The fact that human voice impersonation is not only widely possible but popular should give you an indication of this. Prosody, intonation, range, etc. - it's all flexible and can be learned and duplicated.
The signals are simple too, because we have to encode and decode them quickly. You may or may not be able to picture and rotate an apple tree in your head, but you can easily read this sentence in the voice of David Attenborough.
Moreover, you can easily fine tune a voice model to fit any other speaker. You can store the unique speaker embeddings in a very thin layer. Zero and few shot unseen sampling can even come close to full reproduction. You can measure this all quantitatively.
Voices are not, and never have been, fingerprints. They're just not that unique.
I think "CYA" is maybe a misleading or overflowery term.
In the idealized world, the legal system is meant to provide an accessible alternative to violence for reconciling disputes, but it's increasingly wielded as an impossibly kafkaesque system meant to maintain corporate power over individuals.
I think "CYA" is an overly-flowery term for the reality that they're blocking every avenue for legal recourse, while a variety of other avenues still exist for which adding friction requires the maintenance of expensive and ongoing costs (owning multiple residences, hiring security, etc.)
(To be clear, I am advocating for a more accessible and level legal system, not for UHC-style violence.)
I'm taking some college courses, and one of them explicitly suggests to keep maybe-not-okay communications off of email so that "you don't expose your company to risks of litigation."
Ah, I see. So, when discussing ways to ensure cuatomers cannot utilize our warranty process, I'll make sure to do so in ways that are not traceable and won't show up in discovery.
> In the idealized world, the legal system is meant to provide an accessible alternative to violence for reconciling disputes, but it's increasingly wielded as an impossibly kafkaesque system meant to maintain corporate power over individuals.
This is an overly flowery way of saying: violence.
The worst of the consequences are the same. People end up dead, destitute, and/or with long-term health consequences and are unable to enjoy the fruits labor in the worst cases. In the milder cases i think i'd prefer a bruise for a week to a huge financial loss.
Per the WSJ article last week, I suspect Mercor's playing in a grey area of contracts. It wasn't just voice.[0]
A lot of people were basically wiretapping themselves AND their businesses!
While a lot of Mercor "contractors" claim Mercor over-reached with data gathering via Insightful, it's kind of smart because people are too afraid to complain too much knowing they'll not only lose their primary job, but also open themselves up to uncapped liability for willful misconduct.
Reminds me of my experience when trying to remove my Airbnb account, they require my ID card scans of both sides. I said fuck it and never touch this company again
> Selling the solution to the problem you caused ought to be illegal.
Most tech solutions are built on the problems they created. This includes phones, cars, computers, every software upgrade, and almost every electronic gadget. You are forced to use them because the world around you is no longer compatible with the way of life that was before the introduction of these tech.
This would eliminate the credit report, monitoring and fixing industry, which would be a good thing.
Court records are public in the US. If creditors want to know if you’ve been in financial trouble, they should check for bankruptcies and lawsuits, not the extrajudicial version of those that the credit reporting companies run based on hearsay.
I remember an AI dataset tool asking candidates to record a 1 minute self intro video for interview purposes in 2022. I was wondering if they were manually watching all of them.
This reminds me of all the new companies that want to "help" you get your public information out of $CORPORATE hands; as if these companies will some how not succumb to either enshittification or breach.
The good thing about the grift economy is it grifts itself, like the turtles!
Author here. Wrote this after watching Lapsus$ post the Mercor archive on their leak site earlier this month. The thing that struck me is the combination: voice samples paired with ID document scans. Most breaches leak one or the other. This one ships a deepfake-ready kit. Tried to keep the writeup practical: what an attacker can actually do with this combo (banking voiceprint bypass, Arup-style video calls, insurance fraud), and a 5-step checklist for the contractors who were in the dump.
Happy to discuss the forensic detection side. AudioSeal
watermarks, AASIST anti-spoofing, and how the detection landscape changes
once voice biometrics start leaking at scale.
Interesting - thanks for the rabbit hole today. ;)
Mercer hasn't released many public statements over the incident. Social media posts aren't necessarily public; but I did find this breach notification sample filed with CA - https://oag.ca.gov/ecrime/databreach/reports/sb24-621099 . I guess we'll see if our legislators finally take data privacy seriously.
> Self-audit your public audio footprint. Search YouTube, podcast directories, and old Zoom recording
This is suggestion #1 on your list of remediation steps for victims, but you didn't provide any information on how anyone would actually do that. How exactly would I search the internet for copies of my voice?
Please don't tell me the solution is giving an embedding of my voice to another third party.
Great question. There's no "reverse voice search" yet the way there is for images — that's genuinely a tool the world needs. In the meantime, the most useful thing is searching your name across YouTube and podcast platforms to map out what's already public. And for Mercor contractors specifically, the California AG
breach notice gives you a solid legal basis to request full deletion. Worth doing today.
HSBC offered voice verification years ago and I just laughed and said nope.
I don’t even use biometrics on apple devices, I use a 6 digit pin.
It was always a stupid idea.
The thing about been willing to trade convenience for security is you get called paranoid and then when the other shoe does drop and you are still doing that you still get called paranoid for the current thing you are not doing that “everyone does”.
One more data point for why sueing companies should lead to CEO getting prison time as well. And ideally invent some kind a of equivalent of pruson for non human persons like organisations.
Because right now the incentive to do what's right are so low. Taking a risk with other's people lives is becomming the norm for companies.
I don't know if it's the reason you imply. In the 70s, there were big debates in Germany about privacy and data storage. They spoke of one's data shadow (Datenschatten). I suspect this word comes from that tradition. The reason the word exists would then be the reflection (Verwaltigung) on WW2.
Yeah, so Germany had a ton of secret police files and of course learned very well what happens when a bunch of people start collecting dossiers.
So yeah, of course they've developed that type of distrust. Americans should have also after the 50-60s paranoia of red scare, black people etc. Instead they just spend a few decades building a anti-social state.
I miss the pre-LLM days when you could make a decent argument that having any unnecessary data was just a liability. Now all anybody thinks is “more data for the AI!”
10+ years ago companies were hoovering up data for ML - trying to find correlations in high-dimensionality data. Mostly the results were garbage but occasionally you hit on a real, unexpected phenomenon.
Nowadays you just throw all the data into a black box and believe whatever it says blindly.
Data can never be stolen, because it is not a physical thing. Data can be copied, and it can be erased - sometimes both happens at the same time. Data can be lost, that is when its last existing copy was erased.
So, they should all just rotate their voices ... right?
I jest but the majority of the "normal" people I know are happy to hand over biometrics because _it's easier_. We need to start branding biometrics as "forever passwords" or something to help people understand just what they're handing over when they validate access to their checking account or enter Disney World or whatever else.
Functionally, biometrics are closer to a username than a password.
Fingerprints, DNA, iris scans, gait patterns, etc. are all something you can't change (much like a permanent account ID) and are constantly being presented to the world (much like an email address). In addition under US law, police can compel presentation of fingerprints, but passwords are protected under the 5th amendment.
That's fair. Though, thinking about it this way, I'd argue they're even more like a permanent API key. Again, messaging completely lost on people who don't spend time worrying these things.
the "it's easier" people operate on a fundamentally different way than you or I. they thrive in the world of plausible deniability and social trust. They almost dont care what happens to them as long as it isnt their fault. And they do not consider putting themselves at risk to be the same as being at fault
in a certain light, it's kind of admirable. they live like the world is the way it should be
One of the problems is that "forever passwords" is a term used positively when I worked in banking, as it was a password that the customer could not forget and would not need support using.
So I could easily see a lot of people viewing this as a positive.
That's a really good point. It lays bare some of my biases when it comes to thinking about and communicating with "normal people" about this sort of thing.
Man that’s pretty shitty that Mercor tricked 40k contractors, and then did a poor job of securing their data. There should be stronger consequences for stuff like this.
What happens now is that a lot of clueless CTO that didn't know about this company now know it's name. So the outcome of this mess is probably more business for Mercor
I mean, just look at what happened to Crowdstrike....
I was floating near some ex agency and GS15 folks yesterday in Houston, they explained to me that the Israeli cybersecurity apparatus has had everyone's voicemails for the last 20 years because they inserted themselves into the supply chain of voicemails somehow or another.
Kind of nuts all the ways audio data can be used now.
If this is real, the bigger issue might not even be the leak itself. It could be that we are quietly moving into a world where voice plus ID is enough to fully impersonate someone, and most systems are still not built for that reality.
I wonder how many of the current text-to-speech ML models have large parts of leaked or "stolen" data in their training data? Almost none of the TTS releases seem to talk about exactly where they get their training data from, for some reason. I also wonder if we'll see an explosion in SOTA TTS in ~6 months from now.
GOOG-411 was "competing" with a strong company (1-800-FREE411) by serving no ads in a category worth ~$3.5B at the time. It was inexplicable at the time, but they did this to get voice samples, way back when. For reasons like that, I expect that this category of training is baked — but I don't have current domain knowledge fwiw.
Not really, Mozilla Common Voice (the ImageNet of speech) is larger than this. Their English database has 3814 hours, 1.6 million sentences, from 100k speakers.
There is also an ugly labor story here. The people labeling and training these systems are often the least protected when the data pipeline itself turns into the attack surface.
>Set up a verbal codeword with family and finance contacts. Pick a phrase that has never been spoken on a recording and never typed in chat. Brief the people who handle money on your behalf. If a call ever asks for a transfer, the codeword is mandatory.
good luck with this. most finance people deal with hundreds to thousands of clients. they obviously cant remember everyones code word. commonly used finance systems arent setup to securely store these codewords. they dont have processes or policies in place to implement or adhere to any sort of codeword verification.
>Rotate where voiceprints are still in use. [...] Do that now, ideally from a new recording in a different acoustic environment than the leaked sample.
would this even have an effect? i have never heard of "rotating" a voice print. isnt the whole point of a voice print that you cant really change it? if simply switching your environment completely changes your voice print, that would make voice prints utterly useless to begin with.
With most US banks, you can ask them to put in a note on your account file for a code word, it will show up anytime the account file is pulled up. Now, whether or not a customer service agent will know to do so is another question. Maybe as attack vectors like this are utilized more often it will become part of their SOP. Or just stop using voice verification. In my experience, even if you pass voice verification, it only grants you access to the account and check balance and txs but still requires information like PIN or a code sent in the app or phone number. There are attack vectors for these as well but not guaranteed.
The other use cases (like calling payroll, etc) likely don’t have the same protections and probably would be more effective.
Someone who has hundreds or thousands of clients presumably couldn't remember every client's voice either, so no meaningful security is lost. They are approximately as secure or insecure as before
>presumably couldn't remember every client's voice either, so no meaningful security is lost
there are automated systems for this already. my bank, isp, etc. use them when you call in to skip the traditional verification steps. this fact is also highlighted in the article.
the problem is that there isnt typically a system in place for setting up or validating code words, so the advice given is not practical to implement.
Yeah seems like nonsense advise. Have a code word that was never recorded? I don’t see how that would tote y anything. Like the point of these systems is they can say stuff you never said convincingly
The idea is that the attacker doesn't know the codeword. If the attacker finds out about the codeword then the attacker could indeed fake it. Hence why you shouldn't say/write it in recordings or chat messages.
The biometric pairing is what makes this particularly bad. A leaked password is recoverable. A leaked voiceprint combined with ID scans is permanent, you can not rotate your voice.
The deeper problem is that most of these companies collected this data because they could, not because they needed it for the core service. 'Datensparsamkeit' is the right frame: the voice samples were a liability sitting on a server waiting for exactly this.
I'm pretty sure Google and Apple already have some decent examples of a LOT of people's voices in concert with other data collation. Google Voice IIRC was bought for audio sampling voicemail in the first place. Not sure if Apple has done similar, but would be more surprised if they didn't... Let alone the voice search options for both.
I've been doing similar things on a different platform because as a uni student the pay is kinda nice, but I limit myself to task without voice/video and just input from mouse/keyboard to do reinforcement learning/data tagging. No way I'm trusting these companies or the companies they contract the work with
Is this post not just an ad for a vibe coded site / product? It adds no new info on the mercor breach and advertises something which I presume has even worse safety practices
I'm curious: if i create an online sample from my voice, might this make it a lot harder for an AI model to identify me if every trainingdata contains my particular voice sample?
Isn’t this going to immediately become daily news?
Half the time I call a company they say “we are recording your voice for security / authentication purposes”.
The companies that do that have all the information on me that they require for me to set up an account, so their data breaches will be just like this one, but 1000x larger.
Can we just fast forward through the part where this works for ID theft, past the firefox age verification plugin that uses these datasets, and even through the part where people in the plugin dataset are digital outcasts (this voice has been used too many times. Want to try another?)
At the end of this dark predictable tunnel, maybe there will be a ban on biometrics for important stuff, a repeal of the age verification laws, and actual privacy legislation with teeth.
Where I live there was a common scam to manipulate voice recordings from phone calls. I was very careful back then with phone calls when I ran my own business. Like 15 years ago. Kinda crazy that any service would use voice recognition today as stated.
im the founder of a company that runs deepfake phishing simulations for enterprises, so biased on this one .. but the operational thing the piece misses is that this is the first widely circulated dump where voice, govt ID and selfie all came from the same onboarding session i.e. most enterprise call center auth still treats those as 3 independent factors ..
The scarier piece is that an attacker pulls a contractor from the dump, finds their employer on linkedin, then calls that companys IT helpdesk for a password reset with the cloned voice.
Great point about the helpdesk vector. The LinkedIn-to-IT-reset path is a brilliant illustration of how social engineering chains work. And you're right that audio is the frontier video deepfake detection has gotten really good, lots of great tools out there. Audio is the next wave, and the teams building solutions
for real-world call quality are going to unlock a massive market. Exciting space to be in.
This kind of event is the best argument against needless data hoarding. But it would help if the law better provided for some kind of consequences for negligence.
40k people are not under thread, I am getting AI contractor job offers every month on UpWork, I am glad I haven't accepted more than one as it is just not worth to do.
You could have seen this coming a mile away. So far I have gotten away with never uploading my ID and/or interacting with one of those companies (though one idiot working for some VC thought it was ok to sign a document on my behalf by uploading my signature!!, never mind a bit of fraud) but it is getting harder and harder. Banks and in some cases even governments forcing you to send data to these operators is a very bad idea. But hey, who ever got hurt by some security theater?
I've had to open a bank account for a company here a few years ago and that was right on the bubble of this happening and they still had an option to come by in person with the proper documentation, which I did, now it is all outsourced.
These companies are the fattest targets and they're run by incompetents. You should assume that anything you give them will eventually be part of some hack.
Because historically that's how it worked, but officials just looked at the document and verified that it was the real thing. Then photocopiers came along and it became normalized to take copies of the documents. Then digital copies happened and that changed things completely when coupled with networking technology. What the officials in charge don't seem to understand is that by making digital copies in networked environments the IDs themselves lost their value completely, after all if the digital copy serves any purpose at all as a stand-in for the original then they have become that original.
Tell us more about that fraud story! Was the person your attorney or accountant? Or just some "smart" person who decided to wisely save time by doing fraud?
It was a fund administrator. I still find it unbelievable that they would so casually do this. And yes, they thought they were very smart... and helpful too...
It feels like a dead end because it's being used wrong. "Is this John's voice?" is the wrong question. "Does this call look like how John normally calls?" is way
more interesting. Same device, same time of day, same way of starting a sentence. That whole pattern is much harder to fake than a voice alone. The authentication isn't dead, it just needs to grow up from a single check into a full picture.
Mercor is the most scummy company out there, run by a bunch of sleazeball 20 somethings who are getting a lot of press as the youngest billionaires in the making.
Fidelity seemed to sign you up for this when you called them on the phone almost automatically. Ridiculous since it was defeated easily in a hacker movie from the 1990s using a tape recorder.
they literally handed over their voice, their face, and their government id to train ai models for peanuts - and now lapsus is sitting on 4tb of 'you' that you can never change like a password
> If you were a Mercor contractor and you believe your voice may already be in circulation, ORAVYS will analyze the first three suspect samples free of charge.
Awesome, if you're a victim of an AI company having your voice, you can help yourself by sending another AI company your voice!
> Audio is never used to train commercial models without explicit consent
I'm sure Mercor has explicit consent as well, legal teams are reasonably good at legally covering their asses with license terms.
The irony runs deeper than the free analysis offer. The whole Mercor contractor relationship was this exact pattern: hand over studio-quality voice recordings and ID scans to get paid for data labeling work that didn't require either. "Explicit consent" was buried in the terms, and people clicked through because they needed the paycheck.
Now 40k people have learned that biometrics aren't passwords. You can't rotate your voice.
> biometrics aren't passwords. You can't rotate your voice.
"My voice is my passport. Verify me."
I have to renew my passport every 10 years or so. How do I do that with my voice? I guess it's time to take some vocal lessons.
32 replies →
> Now 40k people have learned that biometrics aren't passwords. You can't rotate your voice.
The problem is that even if you know that, you still get bombarded by banking apps promising "biometrics are more secure than passwords, switch now!"
I doubt 1% of the 40k will learn anything.
also this took me way too long to realize it had nothing to do with warhammer.
> Now 40k people have learned that biometrics aren't passwords. You can't rotate your voice.
Voices aren't strong.
There just aren't that many unique characteristic parameters behind a voice - it's largely dictated by an evolutionary shared shared larynx and vocal tract. They aren't fingerprints.
The fact that human voice impersonation is not only widely possible but popular should give you an indication of this. Prosody, intonation, range, etc. - it's all flexible and can be learned and duplicated.
The signals are simple too, because we have to encode and decode them quickly. You may or may not be able to picture and rotate an apple tree in your head, but you can easily read this sentence in the voice of David Attenborough.
Moreover, you can easily fine tune a voice model to fit any other speaker. You can store the unique speaker embeddings in a very thin layer. Zero and few shot unseen sampling can even come close to full reproduction. You can measure this all quantitatively.
Voices are not, and never have been, fingerprints. They're just not that unique.
1 reply →
I think "CYA" is maybe a misleading or overflowery term.
In the idealized world, the legal system is meant to provide an accessible alternative to violence for reconciling disputes, but it's increasingly wielded as an impossibly kafkaesque system meant to maintain corporate power over individuals.
I think "CYA" is an overly-flowery term for the reality that they're blocking every avenue for legal recourse, while a variety of other avenues still exist for which adding friction requires the maintenance of expensive and ongoing costs (owning multiple residences, hiring security, etc.)
(To be clear, I am advocating for a more accessible and level legal system, not for UHC-style violence.)
I'm taking some college courses, and one of them explicitly suggests to keep maybe-not-okay communications off of email so that "you don't expose your company to risks of litigation."
Ah, I see. So, when discussing ways to ensure cuatomers cannot utilize our warranty process, I'll make sure to do so in ways that are not traceable and won't show up in discovery.
11 replies →
> In the idealized world, the legal system is meant to provide an accessible alternative to violence for reconciling disputes, but it's increasingly wielded as an impossibly kafkaesque system meant to maintain corporate power over individuals.
This is an overly flowery way of saying: violence.
The worst of the consequences are the same. People end up dead, destitute, and/or with long-term health consequences and are unable to enjoy the fruits labor in the worst cases. In the milder cases i think i'd prefer a bruise for a week to a huge financial loss.
2 replies →
Per the WSJ article last week, I suspect Mercor's playing in a grey area of contracts. It wasn't just voice.[0]
A lot of people were basically wiretapping themselves AND their businesses!
While a lot of Mercor "contractors" claim Mercor over-reached with data gathering via Insightful, it's kind of smart because people are too afraid to complain too much knowing they'll not only lose their primary job, but also open themselves up to uncapped liability for willful misconduct.
[0] https://www.wsj.com/tech/ai/mercor-ai-startup-personal-data-...
Reminds me of my experience when trying to remove my Airbnb account, they require my ID card scans of both sides. I said fuck it and never touch this company again
This reminds me of those identity theft settlements, where you need to prove your identity to claim the reward
Has your identity been stolen? Try our free credit monitoring for a month!
Selling the solution to the problem you caused ought to be illegal.
> Selling the solution to the problem you caused ought to be illegal.
Most tech solutions are built on the problems they created. This includes phones, cars, computers, every software upgrade, and almost every electronic gadget. You are forced to use them because the world around you is no longer compatible with the way of life that was before the introduction of these tech.
4 replies →
This would eliminate the credit report, monitoring and fixing industry, which would be a good thing.
Court records are public in the US. If creditors want to know if you’ve been in financial trouble, they should check for bankruptcies and lawsuits, not the extrajudicial version of those that the credit reporting companies run based on hearsay.
1 reply →
I remember an AI dataset tool asking candidates to record a 1 minute self intro video for interview purposes in 2022. I was wondering if they were manually watching all of them.
This reminds me of all the new companies that want to "help" you get your public information out of $CORPORATE hands; as if these companies will some how not succumb to either enshittification or breach.
The good thing about the grift economy is it grifts itself, like the turtles!
[dead]
Author here. Wrote this after watching Lapsus$ post the Mercor archive on their leak site earlier this month. The thing that struck me is the combination: voice samples paired with ID document scans. Most breaches leak one or the other. This one ships a deepfake-ready kit. Tried to keep the writeup practical: what an attacker can actually do with this combo (banking voiceprint bypass, Arup-style video calls, insurance fraud), and a 5-step checklist for the contractors who were in the dump.
Interesting - thanks for the rabbit hole today. ;)
Mercer hasn't released many public statements over the incident. Social media posts aren't necessarily public; but I did find this breach notification sample filed with CA - https://oag.ca.gov/ecrime/databreach/reports/sb24-621099 . I guess we'll see if our legislators finally take data privacy seriously.
Didn't this happen three weeks ago?
Mercor has definitely released statements with boilerplate "investigations are underway."
> Self-audit your public audio footprint. Search YouTube, podcast directories, and old Zoom recording
This is suggestion #1 on your list of remediation steps for victims, but you didn't provide any information on how anyone would actually do that. How exactly would I search the internet for copies of my voice?
Please don't tell me the solution is giving an embedding of my voice to another third party.
Great question. There's no "reverse voice search" yet the way there is for images — that's genuinely a tool the world needs. In the meantime, the most useful thing is searching your name across YouTube and podcast platforms to map out what's already public. And for Mercor contractors specifically, the California AG breach notice gives you a solid legal basis to request full deletion. Worth doing today.
1 reply →
HSBC offered voice verification years ago and I just laughed and said nope.
I don’t even use biometrics on apple devices, I use a 6 digit pin.
It was always a stupid idea.
The thing about been willing to trade convenience for security is you get called paranoid and then when the other shoe does drop and you are still doing that you still get called paranoid for the current thing you are not doing that “everyone does”.
Paraphrasing Franklin and Churchill, those who trade some security for some convenience may soon find themselves possessed of neither at all.
[dead]
[dead]
One more data point for why sueing companies should lead to CEO getting prison time as well. And ideally invent some kind a of equivalent of pruson for non human persons like organisations.
Because right now the incentive to do what's right are so low. Taking a risk with other's people lives is becomming the norm for companies.
The only data that cannot be stolen or leaked is data that doesn't exist. Hard lesson for both users and companies.
Germans (because of course) have a word for this: "Datensparsamkeit". Being frugal with your data.
Do Germans have lots of words or just a lack of spaces?
> Germans (because of course)
I don't know if it's the reason you imply. In the 70s, there were big debates in Germany about privacy and data storage. They spoke of one's data shadow (Datenschatten). I suspect this word comes from that tradition. The reason the word exists would then be the reflection (Verwaltigung) on WW2.
I took the "because of course" to be about having a word for everything - a stereotypical idea about the German language.
6 replies →
The Stasi would be the obvious cultural context.
In the US of course the government buys this sort of information legally from corporations.
4 replies →
Germany resisted Google Street View until 2023, which was something I thought was very impressive.
Love it, also love how Datenschatten can also imply that it disappears when someone shines light on it
1 reply →
Yeah, so Germany had a ton of secret police files and of course learned very well what happens when a bunch of people start collecting dossiers.
So yeah, of course they've developed that type of distrust. Americans should have also after the 50-60s paranoia of red scare, black people etc. Instead they just spend a few decades building a anti-social state.
I miss the pre-LLM days when you could make a decent argument that having any unnecessary data was just a liability. Now all anybody thinks is “more data for the AI!”
10+ years ago companies were hoovering up data for ML - trying to find correlations in high-dimensionality data. Mostly the results were garbage but occasionally you hit on a real, unexpected phenomenon.
Nowadays you just throw all the data into a black box and believe whatever it says blindly.
Were you not around for the Big Data heyday a decade ago?
11 replies →
Data hoarding predates LLMs. There where other machine learning methods which also needed data for training.
9 replies →
Data can never be stolen, because it is not a physical thing. Data can be copied, and it can be erased - sometimes both happens at the same time. Data can be lost, that is when its last existing copy was erased.
The use of "steal" for non-physical things pre-dates the use of "data" in the modern sense [1]. Policing language incorrectly is not reasonable.
[0] https://www.opensourceshakespeare.org/views/plays/play_view....
[1] https://www.etymonline.com/word/data
Money is not a physical thing.
pedantic and true. What was stolen was not data, but future revenue based on exclusive access to that data.
1 reply →
[dead]
Or you could put it in a box with no connection to the internet.
Introducing… The Hooli Box!
Data that is publicly available also can't be stolen or leaked. Nobody can steal Mozilla's common voice dataset.
> The only data that cannot be stolen or leaked is data that doesn't exist. Hard lesson for both users and companies.
Except no company is learning this lesson.
The enterprise threat model includes "our own users", and the modus operandi is to maintain as much information on that threat as possible.
The only winning move is not to play.
Seems a bit like blaming the victim? Your voice (like DNA) is kind of ambient data that's hard to hide.
So, they should all just rotate their voices ... right?
I jest but the majority of the "normal" people I know are happy to hand over biometrics because _it's easier_. We need to start branding biometrics as "forever passwords" or something to help people understand just what they're handing over when they validate access to their checking account or enter Disney World or whatever else.
Functionally, biometrics are closer to a username than a password.
Fingerprints, DNA, iris scans, gait patterns, etc. are all something you can't change (much like a permanent account ID) and are constantly being presented to the world (much like an email address). In addition under US law, police can compel presentation of fingerprints, but passwords are protected under the 5th amendment.
That's fair. Though, thinking about it this way, I'd argue they're even more like a permanent API key. Again, messaging completely lost on people who don't spend time worrying these things.
the "it's easier" people operate on a fundamentally different way than you or I. they thrive in the world of plausible deniability and social trust. They almost dont care what happens to them as long as it isnt their fault. And they do not consider putting themselves at risk to be the same as being at fault
in a certain light, it's kind of admirable. they live like the world is the way it should be
That’s HN users towards politics and the environment. Sitting smugly with their yubikey and encrypted laptop while the world around them crumbles.
1 reply →
One of the problems is that "forever passwords" is a term used positively when I worked in banking, as it was a password that the customer could not forget and would not need support using.
So I could easily see a lot of people viewing this as a positive.
That's a really good point. It lays bare some of my biases when it comes to thinking about and communicating with "normal people" about this sort of thing.
3 replies →
Man that’s pretty shitty that Mercor tricked 40k contractors, and then did a poor job of securing their data. There should be stronger consequences for stuff like this.
What happens now is that a lot of clueless CTO that didn't know about this company now know it's name. So the outcome of this mess is probably more business for Mercor
I mean, just look at what happened to Crowdstrike....
Mercor has around 5 customers that make up 95% of its revenue. Anybody who needs to know about them already does.
At minimum, collecting voiceprints should come with much stricter consent, retention and security requirements than ordinary "training data"
It more looks like the purpose of such company was to steal such data.
Look at their privacy policies. It absolutely is. They are harvesting video, voice, and much more.
> What does an attacker actually do with thirty seconds of someone's clean read voice plus a scan of their driver's license?
I could think of quite a few things. I know that my bank and brokerage use voice ID.
I was floating near some ex agency and GS15 folks yesterday in Houston, they explained to me that the Israeli cybersecurity apparatus has had everyone's voicemails for the last 20 years because they inserted themselves into the supply chain of voicemails somehow or another.
Kind of nuts all the ways audio data can be used now.
A few Israeli companies supply the software used to record phone calls when you call customer service.
If this is real, the bigger issue might not even be the leak itself. It could be that we are quietly moving into a world where voice plus ID is enough to fully impersonate someone, and most systems are still not built for that reality.
I wonder how many of the current text-to-speech ML models have large parts of leaked or "stolen" data in their training data? Almost none of the TTS releases seem to talk about exactly where they get their training data from, for some reason. I also wonder if we'll see an explosion in SOTA TTS in ~6 months from now.
GOOG-411 was "competing" with a strong company (1-800-FREE411) by serving no ads in a category worth ~$3.5B at the time. It was inexplicable at the time, but they did this to get voice samples, way back when. For reasons like that, I expect that this category of training is baked — but I don't have current domain knowledge fwiw.
It's already there. And keeps moving.
Even have a nice UI on top.
https://voicebox.sh/
Not really, Mozilla Common Voice (the ImageNet of speech) is larger than this. Their English database has 3814 hours, 1.6 million sentences, from 100k speakers.
https://commonvoice.mozilla.org/en/languages
Yep, the silence around provenance is probably the most suspicious part
There is also an ugly labor story here. The people labeling and training these systems are often the least protected when the data pipeline itself turns into the attack surface.
>Set up a verbal codeword with family and finance contacts. Pick a phrase that has never been spoken on a recording and never typed in chat. Brief the people who handle money on your behalf. If a call ever asks for a transfer, the codeword is mandatory.
good luck with this. most finance people deal with hundreds to thousands of clients. they obviously cant remember everyones code word. commonly used finance systems arent setup to securely store these codewords. they dont have processes or policies in place to implement or adhere to any sort of codeword verification.
>Rotate where voiceprints are still in use. [...] Do that now, ideally from a new recording in a different acoustic environment than the leaked sample.
would this even have an effect? i have never heard of "rotating" a voice print. isnt the whole point of a voice print that you cant really change it? if simply switching your environment completely changes your voice print, that would make voice prints utterly useless to begin with.
With most US banks, you can ask them to put in a note on your account file for a code word, it will show up anytime the account file is pulled up. Now, whether or not a customer service agent will know to do so is another question. Maybe as attack vectors like this are utilized more often it will become part of their SOP. Or just stop using voice verification. In my experience, even if you pass voice verification, it only grants you access to the account and check balance and txs but still requires information like PIN or a code sent in the app or phone number. There are attack vectors for these as well but not guaranteed.
The other use cases (like calling payroll, etc) likely don’t have the same protections and probably would be more effective.
Someone who has hundreds or thousands of clients presumably couldn't remember every client's voice either, so no meaningful security is lost. They are approximately as secure or insecure as before
>presumably couldn't remember every client's voice either, so no meaningful security is lost
there are automated systems for this already. my bank, isp, etc. use them when you call in to skip the traditional verification steps. this fact is also highlighted in the article.
the problem is that there isnt typically a system in place for setting up or validating code words, so the advice given is not practical to implement.
Yeah seems like nonsense advise. Have a code word that was never recorded? I don’t see how that would tote y anything. Like the point of these systems is they can say stuff you never said convincingly
The idea is that the attacker doesn't know the codeword. If the attacker finds out about the codeword then the attacker could indeed fake it. Hence why you shouldn't say/write it in recordings or chat messages.
The biometric pairing is what makes this particularly bad. A leaked password is recoverable. A leaked voiceprint combined with ID scans is permanent, you can not rotate your voice.
The deeper problem is that most of these companies collected this data because they could, not because they needed it for the core service. 'Datensparsamkeit' is the right frame: the voice samples were a liability sitting on a server waiting for exactly this.
[dead]
I'm pretty sure Google and Apple already have some decent examples of a LOT of people's voices in concert with other data collation. Google Voice IIRC was bought for audio sampling voicemail in the first place. Not sure if Apple has done similar, but would be more surprised if they didn't... Let alone the voice search options for both.
> How to check if your voice is being misused
I love that the answer here is basically.. - you don't -
But maybe mitigate at unreasonable personal costs.
How about services simply stop taking public information as proof of identity?
I've been doing similar things on a different platform because as a uni student the pay is kinda nice, but I limit myself to task without voice/video and just input from mouse/keyboard to do reinforcement learning/data tagging. No way I'm trusting these companies or the companies they contract the work with
Is this post not just an ad for a vibe coded site / product? It adds no new info on the mercor breach and advertises something which I presume has even worse safety practices
I'm curious: if i create an online sample from my voice, might this make it a lot harder for an AI model to identify me if every trainingdata contains my particular voice sample?
I saw the red flags immediately when I stumbled across them a year ago maybe. I'm really not surprised.
Isn’t this going to immediately become daily news?
Half the time I call a company they say “we are recording your voice for security / authentication purposes”.
The companies that do that have all the information on me that they require for me to set up an account, so their data breaches will be just like this one, but 1000x larger.
Can we just fast forward through the part where this works for ID theft, past the firefox age verification plugin that uses these datasets, and even through the part where people in the plugin dataset are digital outcasts (this voice has been used too many times. Want to try another?)
At the end of this dark predictable tunnel, maybe there will be a ban on biometrics for important stuff, a repeal of the age verification laws, and actual privacy legislation with teeth.
Where I live there was a common scam to manipulate voice recordings from phone calls. I was very careful back then with phone calls when I ran my own business. Like 15 years ago. Kinda crazy that any service would use voice recognition today as stated.
im the founder of a company that runs deepfake phishing simulations for enterprises, so biased on this one .. but the operational thing the piece misses is that this is the first widely circulated dump where voice, govt ID and selfie all came from the same onboarding session i.e. most enterprise call center auth still treats those as 3 independent factors ..
The scarier piece is that an attacker pulls a contractor from the dump, finds their employer on linkedin, then calls that companys IT helpdesk for a password reset with the cloned voice.
Fwiw we put up a free realtime face swap demo a while back at https://www.callstrike.ai/deepfake-security-training .. worth a look if you want to actually feel how trivial this has gotten.
Great point about the helpdesk vector. The LinkedIn-to-IT-reset path is a brilliant illustration of how social engineering chains work. And you're right that audio is the frontier video deepfake detection has gotten really good, lots of great tools out there. Audio is the next wave, and the teams building solutions for real-world call quality are going to unlock a massive market. Exciting space to be in.
This kind of event is the best argument against needless data hoarding. But it would help if the law better provided for some kind of consequences for negligence.
40k people are not under thread, I am getting AI contractor job offers every month on UpWork, I am glad I haven't accepted more than one as it is just not worth to do.
You could have seen this coming a mile away. So far I have gotten away with never uploading my ID and/or interacting with one of those companies (though one idiot working for some VC thought it was ok to sign a document on my behalf by uploading my signature!!, never mind a bit of fraud) but it is getting harder and harder. Banks and in some cases even governments forcing you to send data to these operators is a very bad idea. But hey, who ever got hurt by some security theater?
I've had to open a bank account for a company here a few years ago and that was right on the bubble of this happening and they still had an option to come by in person with the proper documentation, which I did, now it is all outsourced.
These companies are the fattest targets and they're run by incompetents. You should assume that anything you give them will eventually be part of some hack.
Why is the ID a hidden secret that can be used for anything regarding security in the first place?
Because historically that's how it worked, but officials just looked at the document and verified that it was the real thing. Then photocopiers came along and it became normalized to take copies of the documents. Then digital copies happened and that changed things completely when coupled with networking technology. What the officials in charge don't seem to understand is that by making digital copies in networked environments the IDs themselves lost their value completely, after all if the digital copy serves any purpose at all as a stand-in for the original then they have become that original.
Tell us more about that fraud story! Was the person your attorney or accountant? Or just some "smart" person who decided to wisely save time by doing fraud?
It was a fund administrator. I still find it unbelievable that they would so casually do this. And yes, they thought they were very smart... and helpful too...
This is exactly why "voice as authentication" feels like a dead end to me
It feels like a dead end because it's being used wrong. "Is this John's voice?" is the wrong question. "Does this call look like how John normally calls?" is way more interesting. Same device, same time of day, same way of starting a sentence. That whole pattern is much harder to fake than a voice alone. The authentication isn't dead, it just needs to grow up from a single check into a full picture.
I love how the check if your affected involves giving a voice sample to whatever the fuck that website is
It's like those have been owned websites. Where you type in your name email and they grab your IP location and anything else to sell it off.
I'm at the point where I might start professionally using a voice changer. I mean what in the world, my guy?
Open Source now?
Now open Chinese models can catch up
Mercor is the most scummy company out there, run by a bunch of sleazeball 20 somethings who are getting a lot of press as the youngest billionaires in the making.
Can't wait for them to crash and burn.
30 under 30 doing 10 to 20 candidates right there.
Youngest totally self-made billionaires.
"My voice is my passport. Verify Me."
:)
HSBC did that. I could never understand that - the exact phrase was in the movie!
Someone probably did it for an internal demo, as a joke. Then people pushed it upwards, until someone clueless approved it.
Fidelity seemed to sign you up for this when you called them on the phone almost automatically. Ridiculous since it was defeated easily in a hacker movie from the 1990s using a tape recorder.
1 reply →
not to be conspiratorial but stolen? or given away...
they literally handed over their voice, their face, and their government id to train ai models for peanuts - and now lapsus is sitting on 4tb of 'you' that you can never change like a password
[dead]
[dead]
[dead]
[dead]
[dead]
[flagged]