I'm planning to spend my evening checking every other Ed25519 implementation I can find to see if this check is missing any where else in the open source ecosystem.
I found several libraries that simply didn't implement the check, but none that implemented in incorrectly in the same way as the vulnerability discussed above.
Did you also check all of the libraries that implement the check differently to libsodium? That's one problem with the near-universal cargo-culting of ref10, it never did any of the checking so everyone has to reinvent it themselves in different ways. It might be useful to have a single known-good check for both x25519 and ed25519 that people could integrate into their own ref10-derived code.
For people not familiar with the size of the mess we're in here, see https://hdevalence.ca/blog/2020-10-04-its-25519am/. There was another study published before then which found that no two implementations used the same checks, and none of them were compliant with RFC 8032, the alleged standard for Ed25519.
I've been iterating on sodium bindings in Lean4 for about four months, and now that I've gotten to Ristretto255 I can see why the author is excited about its potential. Ristretto is a tightly designed API that allows me to build arbitrary polynomials on Curve25519 and I've been having a blast tinkering and experimenting with it! If the author by chance reads this, just want to say thank you for your work!
The bindings are set and have a monadic interface, but there's some abstractions that still need refining/iterating: mostly I want to be able to formalize keyboard input and eventually build a tactic framework for zero-knowledge proofs.
Libsodium’s goal was to expose APIs to perform operations, not low-level functions. Users shouldn’t even have to know or care about what algorithms are used internally. This is how I’ve always viewed libsodium.
...
Over the years, people started using these low-level functions directly. Libsodium started to be used as a toolkit of algorithms and low-level primitives.
That is interesting to see the common fallacy of what we think users want versus what they really want.
The important point is to be able to recognize that and not coerce users into using your project only how you envisioned it and only like that. Some projects are failure on that count having switched on dictatorial direction on that aspect.
>That is interesting to see the common fallacy of what we think users want versus what they really want.
Or a fallacy of what users think they want versus what they really want.
Non-cryptographers shouldn't use cryptographic primitives directly in security critical coffee paths. Libsodium tried to protect users from themselves in that regard. I think that's a worthy goal - library should try to make it impossible to use it incorrectly, which means high level primitives.
> The important point is to be able to recognize that and not coerce users into using your project only how you envisioned it and only like that. Some projects are failure on that count having switched on dictatorial direction on that aspect.
There is certainly a balance there. If every function inside your code is now considered part of your API contract, almost anything is a breaking change and you can basically forget about ever meaningfully refactoring that codebase.
Many times making things private or marking them as internal-only is the right call.
I'm not really intimate enough with libsodium to judge if they made the right cut there or not in hindsight.
> and you can basically forget about ever meaningfully refactoring that codebase.
Ummm why? Breaking changes aren't the end of the world? Deprecate and communicate clearly and people are usually fine with them (if it's meaningful progress instead of churn).
I wrote a C++ implementation of the Framework for Integrated Test (FIT) called CeeFIT, and I was really proud of the way it registered fixtures at compile time.
Anyhow, I was surprised that more than one user was using CeeFIT as a sort of batch runner for C++ code, feeding in rows tabular data and executing it against their code. There were a couple bugs I had to fix to support their use cases.
Some of the most successful products were originally intended for a completely different use case. R7 rockets, Viagra, Hugging Face. The ability to pivot - and to recognize when to pivot - is what makes or breaks.
Subtle but important bug. This is a good example of how “is valid” checks in crypto are rarely as simple as they sound. Accepting points outside the prime-order subgroup can quietly undermine higher-level assumptions, even if no immediate exploit is obvious. Also a reminder that low-level primitives tend to be reused far more widely than intended, so small validation gaps can have surprisingly large blast radii.
Do note thought that X25519 and Ed25519 were designed so they wouldn’t need those checks at all. It’s only when you’re trying to design fancier protocols on top of Curve25519 or Edwards25519 that you can run into subgroup issues.
And for those use cases, I personally try my best to just reproject everything back into the prime order subgroup whenever possible. Monocypher has a number of such fancy functions:
The dirty functions explicitly produce public keys that cover the entire curve, so that random such keys are truly indistinguishable from random when converted with `crypto_elligator_rev()`. But instead of just removing the clamp operation, I instead add random low-order point, so that when we later use the point in an X25519 key exchange, the shared secret is exactly the same as it would have been for a genuine X255119 key.
That’s where I thank DJB for designing a key exchange protocol that project the shared secret to the prime order subgroup, even when the public key it processes is not. The original intent may have been to make checks easier (low order keys all end up yielding zero), but a nice side effect is how it enabled a nice API for Mike Hamburg’s Elligator2.
> Accepting points outside the prime-order subgroup can quietly undermine higher-level assumptions, even if no immediate exploit is obvious.
If on the other hand we can prove that all computed results are low-order-component-independent (as is the case for X25519), then we know for sure we’re safe. In the end, Ristretto is only really needed when we can’t tweak the protocol to safely reproject to the prime order subgroup.
Don’t get me wrong, having a prime order group abstraction does help. But if someone is qualified to design a protocol that may require this, they’re qualified to try and make it work with a non-trivial cofactor as well — that, or prove it cannot be done.
I work for a big company (Apple) but I have no idea who Frank is, nor how to sponsor them; and even if I knew them and how to sponsor them, the money would come directly from my pocket instead of Apple’s banking account.
If libsodium is useful to you, please keep in mind that it is maintained by one person, for free, in time I could spend with my family or on other projects. The best way to help the project would be to consider sponsoring it, which helps me dedicate more time to improving it and making it great for everyone, for many more years to come.
Maybe you don't know this but Apple has a donation-matching program. If you make donations to non-profits through some special internal mechanism, the company will send a donation of equal value (up to some limit). If I recall correctly the limit is 30K USD per person.
This also affected the PHP library, sodium_compat. https://github.com/FriendsOfPHP/security-advisories/pull/756
I'm planning to spend my evening checking every other Ed25519 implementation I can find to see if this check is missing any where else in the open source ecosystem.
I found several libraries that simply didn't implement the check, but none that implemented in incorrectly in the same way as the vulnerability discussed above.
If you didn't receive an email from me, either your implementation isn't listed on https://ianix.com/pub/ed25519-deployment.html, I somehow missed it, or you're safe.
Did you also check all of the libraries that implement the check differently to libsodium? That's one problem with the near-universal cargo-culting of ref10, it never did any of the checking so everyone has to reinvent it themselves in different ways. It might be useful to have a single known-good check for both x25519 and ed25519 that people could integrate into their own ref10-derived code.
For people not familiar with the size of the mess we're in here, see https://hdevalence.ca/blog/2020-10-04-its-25519am/. There was another study published before then which found that no two implementations used the same checks, and none of them were compliant with RFC 8032, the alleged standard for Ed25519.
1 reply →
Thank you for your work on free software.
Thank you for your work on open source.
I've been iterating on sodium bindings in Lean4 for about four months, and now that I've gotten to Ristretto255 I can see why the author is excited about its potential. Ristretto is a tightly designed API that allows me to build arbitrary polynomials on Curve25519 and I've been having a blast tinkering and experimenting with it! If the author by chance reads this, just want to say thank you for your work!
You have a public repo of this?
Yes: https://github.com/rj-calvin/sodium
The bindings are set and have a monadic interface, but there's some abstractions that still need refining/iterating: mostly I want to be able to formalize keyboard input and eventually build a tactic framework for zero-knowledge proofs.
That is interesting to see the common fallacy of what we think users want versus what they really want.
The important point is to be able to recognize that and not coerce users into using your project only how you envisioned it and only like that. Some projects are failure on that count having switched on dictatorial direction on that aspect.
>That is interesting to see the common fallacy of what we think users want versus what they really want.
Or a fallacy of what users think they want versus what they really want.
Non-cryptographers shouldn't use cryptographic primitives directly in security critical coffee paths. Libsodium tried to protect users from themselves in that regard. I think that's a worthy goal - library should try to make it impossible to use it incorrectly, which means high level primitives.
See also one of my favorite cryptographic essays, "If You're Typing The Letters A-E-S Into Your Code, You're Doing It Wrong" https://people.eecs.berkeley.edu/~daw/teaching/cs261-f12/mis...
> The important point is to be able to recognize that and not coerce users into using your project only how you envisioned it and only like that. Some projects are failure on that count having switched on dictatorial direction on that aspect.
There is certainly a balance there. If every function inside your code is now considered part of your API contract, almost anything is a breaking change and you can basically forget about ever meaningfully refactoring that codebase.
Many times making things private or marking them as internal-only is the right call.
I'm not really intimate enough with libsodium to judge if they made the right cut there or not in hindsight.
> and you can basically forget about ever meaningfully refactoring that codebase.
Ummm why? Breaking changes aren't the end of the world? Deprecate and communicate clearly and people are usually fine with them (if it's meaningful progress instead of churn).
2 replies →
I wrote a C++ implementation of the Framework for Integrated Test (FIT) called CeeFIT, and I was really proud of the way it registered fixtures at compile time.
Anyhow, I was surprised that more than one user was using CeeFIT as a sort of batch runner for C++ code, feeding in rows tabular data and executing it against their code. There were a couple bugs I had to fix to support their use cases.
I was just happy to have users.
Some of the most successful products were originally intended for a completely different use case. R7 rockets, Viagra, Hugging Face. The ability to pivot - and to recognize when to pivot - is what makes or breaks.
[flagged]
Subtle but important bug. This is a good example of how “is valid” checks in crypto are rarely as simple as they sound. Accepting points outside the prime-order subgroup can quietly undermine higher-level assumptions, even if no immediate exploit is obvious. Also a reminder that low-level primitives tend to be reused far more widely than intended, so small validation gaps can have surprisingly large blast radii.
Do note thought that X25519 and Ed25519 were designed so they wouldn’t need those checks at all. It’s only when you’re trying to design fancier protocols on top of Curve25519 or Edwards25519 that you can run into subgroup issues.
And for those use cases, I personally try my best to just reproject everything back into the prime order subgroup whenever possible. Monocypher has a number of such fancy functions:
The dirty functions explicitly produce public keys that cover the entire curve, so that random such keys are truly indistinguishable from random when converted with `crypto_elligator_rev()`. But instead of just removing the clamp operation, I instead add random low-order point, so that when we later use the point in an X25519 key exchange, the shared secret is exactly the same as it would have been for a genuine X255119 key.
That’s where I thank DJB for designing a key exchange protocol that project the shared secret to the prime order subgroup, even when the public key it processes is not. The original intent may have been to make checks easier (low order keys all end up yielding zero), but a nice side effect is how it enabled a nice API for Mike Hamburg’s Elligator2.
> Accepting points outside the prime-order subgroup can quietly undermine higher-level assumptions, even if no immediate exploit is obvious.
If on the other hand we can prove that all computed results are low-order-component-independent (as is the case for X25519), then we know for sure we’re safe. In the end, Ristretto is only really needed when we can’t tweak the protocol to safely reproject to the prime order subgroup.
Don’t get me wrong, having a prime order group abstraction does help. But if someone is qualified to design a protocol that may require this, they’re qualified to try and make it work with a non-trivial cofactor as well — that, or prove it cannot be done.
If you work for a big company, consider trying to get Frank sponsored by your company.
I work for a big company (Apple) but I have no idea who Frank is, nor how to sponsor them; and even if I knew them and how to sponsor them, the money would come directly from my pocket instead of Apple’s banking account.
From the article:
The "sponsoring it" links to https://opencollective.com/libsodium/contribute
Hope that helps.
15 replies →
Maybe you don't know this but Apple has a donation-matching program. If you make donations to non-profits through some special internal mechanism, the company will send a donation of equal value (up to some limit). If I recall correctly the limit is 30K USD per person.
8 replies →
Is libnacl affected
I use software compiled with libnacl every day but none compiled with "libsodium"
Such a great library. Thank you Frank Denis