Thanks for clarifying, good to see you're using a decent hashing algorithm :)
I'm still a little unsure of how you are able to know some users had their password compromised. Is it a simple case of finding successful log in attempts from the same IP address as attack?
That's good to hear. As a future suggestion to anyone else who finds themselves in this unfortunate situation - including some technical granularity in your press release can go miles in offering reassurance to your technical audience/users.
So what do you do past this point? I know you can probably rough out how much time it would take to find hash collisions and ask as your users to change their passwords before that amount of time elapses, but past that point, can't you no longer assume that it's the actual user logging in to change their password?
Can I suggest you also share what parameters you use with scrypt? Scrypt is parametric and you can choose weak or strong choices for parameters depending on how long you want to spend validating passwords.
»What hashing scheme do you use?« does not matter for most users.
Most users use weak passwords and a substantial part of this passwords is easy to recover using a dictionary attack. It does not really matter if you use MD5, SHA1, SHA2, HMAC, PBKDF2, bcrypt, scrypt or whatever, nor does it matter if you use no salt, the same salt for all users or a unique salt per user. Even for PBKDF2, bcrypt and scypt the cost factor will - for practical reasons - usually not be large enough to mitigate dictionary attacks using a few thousand of the most common passwords. Therefore weak passwords are compromised regardless of the used hashing scheme. And because especially users with weak passwords tend to reuse the password for different accounts many other accounts are compromised, too.
A user caring about security will not reuse passwords for different accounts and this alone reduces the impact of the event by a huge amount. Further a strong password alone makes it very unlikely that attackers will recover the password even if only unsalted MD5 is used for hashing. Therefore - unless the password is stored in plain text - it is highly unlikely that an attacker will be able to access an account protected by a strong password.
I definitely don't want to argue that using unsalted MD5 is okay - it is not - but for the average user the difference between a weak and a strong hashing scheme is not as large as one would naively expect. Strong hashing schemes will especially protect users using infrequent dictionary words or medium length hard passwords because the additional computation power required to perform a dictionary or brute force attack will force the attackers to use smaller dictionaries and shorter passwords.
Finally storing passwords may benefit from security through obscurity. If the attacker is unable to figure out the used hashing scheme he will be unable to perform a dictionary or brute force attack. This does not mean everyone should come up with there own hashing scheme - this would do MUCH more harm than good - but, for example, using a unknown random - 294,897 instead of 300,000 - cost factor and keeping it secret or adding a second secret salt buried deep in the code to the salt stored together with the username and hash will make it quite a bit harder for the attacker to perform an attack unless they got the information from an insider or were able to steal your code or binaries.
Using an in-code secret key (as opposed to the not-secret-by-necessity salt) is commonly referred to as a pepper. It improves security in the cases when an attacker has access to your database but not to your code or filesystem.
Figuring out the hashing scheme used for a given hash is frequently trivial. All an attacker needs to do is hijack his own hashed password and salt and then run combinations of common hashes with salting patterns until he gets a hit. This is going to be hundreds of combinations to test on the high end, and will generally yield results very easily.
> It does not really matter if you use ... a unique salt per user.
I agree with the other points that you make on this aspect but I do not quite understand this particular point (quoted above) can be true. If you use a strong unique salt for each user's password, then you are padding the length of actual password hashed and thereby effectively reducing the possibility of a successful dictionary attack to virtually zero. If this is so, then how could one mount a successful dictionary attack ?
> How long was the salt? AFIK, MD5 hashes with an insufficiently long salt can be bruteforced with open sourced CUDA setups.
The length of the salt has little impact on security beyond 16 bits or so, where it's still feasible to generate rainbow tables for all salts.
If you're storing plain hashes, it doesn't really matter whether it's MD5, SHA-1 or SHA-256 - the work required for a brute-force attack is largely the same. The next step up would be using a key stretching algorithm like PBKDF2 or bcrypt.
kpumunk and others, please note that http://www.scribd.com/password/check leaks information about whether or not a particular email address is in your system. It's hardly a critical weakness or anything, but just an FYI and something you may wish to reevaluate.
Edit: thanks for the quick fix! I would advise that a good compromise would be to change the current message from
"Good news - your password was not among those compromised. You do not need to take any action at this time."
to
"Good news - if this is the email your Scribd account is associated with, your password was not among those compromised. You do not need to take any action at this time."
Small difference, but the lack of definiteness to the response will be enough to make me think of double-checking my email (in my case, it was an email address I haven't used in years - I'm one of the first Scribd users!)
This seems to have become a common occurrence. Just curious about a couple of things :
a) How easy/hard is it for the hashed passwords to be cracked by the hacker?
b) How are hackers getting access to emails and password data so often? You would think these large sites have enough layers of security to prevent this from happening. Is it social engineering or common loopholes in these systems?
b) Because it's hard to secure a corporate infrastructure, (i.e., making a good set of usability/security trade-offs), and there's no such thing as perfect security. A web application that's been audited can still have a security flaw that enables disclosure of authentication data. Or an employee's machine might be compromised, leading to a compromise of corporate infrastructure or data that he/she is able to access.
Does the reference to similar incidents this year suggest this was a watering hole attack targeting a Java vulnerability? It would be nice to know, just generally, what vectors were used here to the extent it was anything novel.
>Even though this information was accessed, the passwords stored by Scribd are encrypted (in technical terms, they are salted and hashed). Most of our users were therefore unaffected by this; however, our analysis shows that a small percentage may have had their passwords compromised.
Could someone explain me what this means: "our analysis shows that a small percentage my have had their password compromised" ?
Do you think this refers to the fact that they might have conducted a statistical analysis which evidenced the fact that _potentially_ a relevant percentage of password could have completely decrypted?
I mean, is this warning just the outcome of a statistical analysis on the possibility that password could have been decrypted?
Or they are still just referring to the fact alone that a small percentage of hashed & salted password have leaked?
I have gone through the "check your email" form. And this was the result:
>We're very sorry to tell you that your Scribd password was among those compromised. If you have used this password on any other services, you should change it immediately.
Compromised here refers to leaked, or potentially decrypted based on some sort of statistical analysis they made?
Thankfully this doesn't read like the typical large corporation email: "...the attacker was able to gain access to all the passwords, which were stored as plaintext in the database." It sad to have to say it, but Kudos to Scribd for actually storing passwords the way they should be stored.
I received this email, but it landed in my spam folder as suspected phishing...
Be careful with this message. Similar messages have been used to steal people's personal information. Unless you trust the sender, don't click on links or reply with personal information.
I thought this was the hackers that got my email off scribd trying to phish my password.
That was my immediate reaction upon seeing it as well, although in this case I noticed that it was sent to me+scribd@example.com, which was indeed the mail that I signed up with. Whilst it's mostly useful for filtering/labelling or figuring out which company sold you to a spam list, it's quite useful in this case as a basic 'yep, not just a scattershot phishing attempt' indicator.
Not foolproof, of course; the people who stole hashes & emails would be the obvious choice to attempt a quick phish, and they now have all the account emails. I wonder if I'll see an uptick in spam...
hehe, same here. My email address was completely randomized, and only used for scribd too (something like f9xl203js@mydomain.com), so I was very sure it was either scribd, or it fell into the wrong hands. When the email ends up in the spam folder and mark as potential phishing as well - my gut feeling was therefore that my email was indeed leaked.
The irony is that the email itself was generated by scribd itself, and not by any spammers.
Props to Scribd, this is the first time I've seen a company include a tool that lets you check if your account might have compromised.
http://www.scribd.com/password/check
Why is email/password more attractive than billing information (credit card or bank/savings account) to hackers? What's the end goal here? I would assume that it's some sort of monetary compensation? With an email and a password, you might have access to someone's banking website, but you have to guess which bank they use and you have to assume that they used the same password. It doesn't seem practical to do a massive scripted attack against a bank's website with a possible list of emails/passwords that may or may not exist on a specific bank's web site.
Now, credit card information on the other hand might be useful. There's a black market for credit card numbers, especially if they include the verification code (CSC/CVV/etc).
If hackers have access to a site's database that contains user authentication data why are we to assume that the hackers didn't have access to payment information or that they didn't have a desire to access it?
Do you understand what salt and hash means? The algorithm may be the same, but the result is going to be different. There are many hashing algorithms, and many ways of salting the hashes. It's not always the same.
Every single time this happens, I immediately wonder: "what was the hashing scheme?"
Like many others before it, Scribd disappoints by not addressing this question. Instead we get this:
Even though this information was accessed, the passwords stored by Scribd are encrypted (in technical terms, they are salted and hashed).
How long was the salt? AFIK, MD5 hashes with an insufficiently long salt can be bruteforced with open sourced CUDA setups.
Further, how did they determine the following?
Most of our users were therefore unaffected by this; however, our analysis shows that a small percentage may have had their passwords compromised.
We use scrypt for passwords hashing. This is modern hard to crack password hashing algorithm.
We do have database access logs, so it was pretty straightforward to identify which users were affected.
You should add this. Savvy people will be positively surprised to see a company actually caring about doing password authentication right.
That is awesome. You should feel comfortable telling people this; it puts you way ahead of the game.
Thanks for clarifying, good to see you're using a decent hashing algorithm :)
I'm still a little unsure of how you are able to know some users had their password compromised. Is it a simple case of finding successful log in attempts from the same IP address as attack?
8 replies →
http://www.scribd.com/password/check thank you for this. now I can run a list of emails against this to see who has scribd account
6 replies →
That's good to hear. As a future suggestion to anyone else who finds themselves in this unfortunate situation - including some technical granularity in your press release can go miles in offering reassurance to your technical audience/users.
4 replies →
So what do you do past this point? I know you can probably rough out how much time it would take to find hash collisions and ask as your users to change their passwords before that amount of time elapses, but past that point, can't you no longer assume that it's the actual user logging in to change their password?
8 replies →
Can I suggest you also share what parameters you use with scrypt? Scrypt is parametric and you can choose weak or strong choices for parameters depending on how long you want to spend validating passwords.
»What hashing scheme do you use?« does not matter for most users.
Most users use weak passwords and a substantial part of this passwords is easy to recover using a dictionary attack. It does not really matter if you use MD5, SHA1, SHA2, HMAC, PBKDF2, bcrypt, scrypt or whatever, nor does it matter if you use no salt, the same salt for all users or a unique salt per user. Even for PBKDF2, bcrypt and scypt the cost factor will - for practical reasons - usually not be large enough to mitigate dictionary attacks using a few thousand of the most common passwords. Therefore weak passwords are compromised regardless of the used hashing scheme. And because especially users with weak passwords tend to reuse the password for different accounts many other accounts are compromised, too.
A user caring about security will not reuse passwords for different accounts and this alone reduces the impact of the event by a huge amount. Further a strong password alone makes it very unlikely that attackers will recover the password even if only unsalted MD5 is used for hashing. Therefore - unless the password is stored in plain text - it is highly unlikely that an attacker will be able to access an account protected by a strong password.
I definitely don't want to argue that using unsalted MD5 is okay - it is not - but for the average user the difference between a weak and a strong hashing scheme is not as large as one would naively expect. Strong hashing schemes will especially protect users using infrequent dictionary words or medium length hard passwords because the additional computation power required to perform a dictionary or brute force attack will force the attackers to use smaller dictionaries and shorter passwords.
Finally storing passwords may benefit from security through obscurity. If the attacker is unable to figure out the used hashing scheme he will be unable to perform a dictionary or brute force attack. This does not mean everyone should come up with there own hashing scheme - this would do MUCH more harm than good - but, for example, using a unknown random - 294,897 instead of 300,000 - cost factor and keeping it secret or adding a second secret salt buried deep in the code to the salt stored together with the username and hash will make it quite a bit harder for the attacker to perform an attack unless they got the information from an insider or were able to steal your code or binaries.
Using an in-code secret key (as opposed to the not-secret-by-necessity salt) is commonly referred to as a pepper. It improves security in the cases when an attacker has access to your database but not to your code or filesystem.
Figuring out the hashing scheme used for a given hash is frequently trivial. All an attacker needs to do is hijack his own hashed password and salt and then run combinations of common hashes with salting patterns until he gets a hit. This is going to be hundreds of combinations to test on the high end, and will generally yield results very easily.
> It does not really matter if you use ... a unique salt per user.
I agree with the other points that you make on this aspect but I do not quite understand this particular point (quoted above) can be true. If you use a strong unique salt for each user's password, then you are padding the length of actual password hashed and thereby effectively reducing the possibility of a successful dictionary attack to virtually zero. If this is so, then how could one mount a successful dictionary attack ?
1 reply →
Back when Evernote was hacked, I got the idea of creating a draft of the kind of response I would prefer in a case like this.
I also intended to write a simple website script that could generate a statement. Things came up, and the gist has gathered dust for a while.
The gist is available here with some example cases listed that companies can learn from, and people are free to provide feedback or spin it off:
http://pygm.us/EwNHanBP
Post your most important feedback in its comments, so other companies reading the gist see it as well.
Companies definitely need to be prepared for full disclosure in the event of a security breach.
> How long was the salt? AFIK, MD5 hashes with an insufficiently long salt can be bruteforced with open sourced CUDA setups.
The length of the salt has little impact on security beyond 16 bits or so, where it's still feasible to generate rainbow tables for all salts.
If you're storing plain hashes, it doesn't really matter whether it's MD5, SHA-1 or SHA-256 - the work required for a brute-force attack is largely the same. The next step up would be using a key stretching algorithm like PBKDF2 or bcrypt.
Why do you want to know the hashing scheme? Isn't it better if nobody knows? :)
No
kpumunk and others, please note that http://www.scribd.com/password/check leaks information about whether or not a particular email address is in your system. It's hardly a critical weakness or anything, but just an FYI and something you may wish to reevaluate.
Edit: thanks for the quick fix! I would advise that a good compromise would be to change the current message from
"Good news - your password was not among those compromised. You do not need to take any action at this time."
to
"Good news - if this is the email your Scribd account is associated with, your password was not among those compromised. You do not need to take any action at this time."
Small difference, but the lack of definiteness to the response will be enough to make me think of double-checking my email (in my case, it was an email address I haven't used in years - I'm one of the first Scribd users!)
Er, what? Every single site I can think of 'leaks' this information through their registration page.
Try and sign up for a new Scribd account with an existing email:
"That email address is already taken; please choose another one"
Try to sign up for a Facebook account with an existing email:
"Sorry, it looks like somebody@somewhere.com belongs to an existing account"
Doesn't really matter, the password reset page leaks the same info and has presumably been up forever:
There is no account registered to mail 'abc@smackfu.com'.
And that's a lot harder to fix.
Why is that harder to fix? Many sites handle it properly:
"If there was an account associated with that email address we have emailed it with instructions."
1 reply →
Thank you for your suggestion. Not a weakness anymore :)
This seems to have become a common occurrence. Just curious about a couple of things : a) How easy/hard is it for the hashed passwords to be cracked by the hacker? b) How are hackers getting access to emails and password data so often? You would think these large sites have enough layers of security to prevent this from happening. Is it social engineering or common loopholes in these systems?
a) Depends on what hashing algorithm is used (and whether it incorporates a salt). http://hashcat.net/oclhashcat-plus/#performance
b) Because it's hard to secure a corporate infrastructure, (i.e., making a good set of usability/security trade-offs), and there's no such thing as perfect security. A web application that's been audited can still have a security flaw that enables disclosure of authentication data. Or an employee's machine might be compromised, leading to a compromise of corporate infrastructure or data that he/she is able to access.
Does the reference to similar incidents this year suggest this was a watering hole attack targeting a Java vulnerability? It would be nice to know, just generally, what vectors were used here to the extent it was anything novel.
>Even though this information was accessed, the passwords stored by Scribd are encrypted (in technical terms, they are salted and hashed). Most of our users were therefore unaffected by this; however, our analysis shows that a small percentage may have had their passwords compromised.
Could someone explain me what this means: "our analysis shows that a small percentage my have had their password compromised" ?
Do you think this refers to the fact that they might have conducted a statistical analysis which evidenced the fact that _potentially_ a relevant percentage of password could have completely decrypted?
I mean, is this warning just the outcome of a statistical analysis on the possibility that password could have been decrypted?
Or they are still just referring to the fact alone that a small percentage of hashed & salted password have leaked?
I have gone through the "check your email" form. And this was the result:
>We're very sorry to tell you that your Scribd password was among those compromised. If you have used this password on any other services, you should change it immediately.
Compromised here refers to leaked, or potentially decrypted based on some sort of statistical analysis they made?
Makes me think they implemented a password logger on login. Not just a database leak but a complete compromise.
Thankfully this doesn't read like the typical large corporation email: "...the attacker was able to gain access to all the passwords, which were stored as plaintext in the database." It sad to have to say it, but Kudos to Scribd for actually storing passwords the way they should be stored.
> (...) corporation email (...)
And even government agencies:
"UK intelligence agency stores passwords in plain text" http://www.zdnet.com/uk-intelligence-agency-stores-passwords...
I received this email, but it landed in my spam folder as suspected phishing...
I thought this was the hackers that got my email off scribd trying to phish my password.
That was my immediate reaction upon seeing it as well, although in this case I noticed that it was sent to me+scribd@example.com, which was indeed the mail that I signed up with. Whilst it's mostly useful for filtering/labelling or figuring out which company sold you to a spam list, it's quite useful in this case as a basic 'yep, not just a scattershot phishing attempt' indicator.
Not foolproof, of course; the people who stole hashes & emails would be the obvious choice to attempt a quick phish, and they now have all the account emails. I wonder if I'll see an uptick in spam...
hehe, same here. My email address was completely randomized, and only used for scribd too (something like f9xl203js@mydomain.com), so I was very sure it was either scribd, or it fell into the wrong hands. When the email ends up in the spam folder and mark as potential phishing as well - my gut feeling was therefore that my email was indeed leaked.
The irony is that the email itself was generated by scribd itself, and not by any spammers.
Props to Scribd, this is the first time I've seen a company include a tool that lets you check if your account might have compromised. http://www.scribd.com/password/check
Hmm, Scribd uses Rails. Possible it was an unpatched Rails exploit?
Why is email/password more attractive than billing information (credit card or bank/savings account) to hackers? What's the end goal here? I would assume that it's some sort of monetary compensation? With an email and a password, you might have access to someone's banking website, but you have to guess which bank they use and you have to assume that they used the same password. It doesn't seem practical to do a massive scripted attack against a bank's website with a possible list of emails/passwords that may or may not exist on a specific bank's web site.
Now, credit card information on the other hand might be useful. There's a black market for credit card numbers, especially if they include the verification code (CSC/CVV/etc).
If hackers have access to a site's database that contains user authentication data why are we to assume that the hackers didn't have access to payment information or that they didn't have a desire to access it?
bad news, it's horrible. I think different website should use different hash with salt to avoid this situation
Do you understand what salt and hash means? The algorithm may be the same, but the result is going to be different. There are many hashing algorithms, and many ways of salting the hashes. It's not always the same.