Comment by LinuxBender
3 years ago
Archive [1]
Their theories are interesting and would explain how they obtained the certs if true. Perhaps the take-away here is to have multiple probes on multiple providers checking all of ones TLS fingerprints and alerting on unknown fingerprints and then checking the certificate transparency log or lack thereof.
openssl s_client -servername news.ycombinator.com -connect news.ycombinator.com:443 < /dev/null 2>/dev/null | openssl x509 -fingerprint -noout -in /dev/stdin
SHA1 Fingerprint=7E:49:BA:40:86:87:B3:39:66:93:94:9E:9C:45:71:85:3C:8D:95:16
[1] - https://archive.ph/C0jYJ [updated]
I've added "Could you prevent or monitor this kind of attack?" section.
There are several indications which could be used to discover the attack from day 1:
* All issued SSL/TLS certificates are subject to certificate transparency. It is worth configuring certificate transparency monitoring, such as Cert Spotter (source on github), which will notify you by email of new certificates issued for your domain names
* Limit validation methods and set exact account identifier which could issue new certificates with Certification Authority Authorization (CAA) Record Extensions for Account URI and Automatic Certificate Management Environment (ACME) Method Binding (RFC 8657) to prevent certificate issue for your domain using other certificate authorities, ACME accounts or validation methods
* Monitor SSL/TLS certificate changes on all your services using external service
* Monitor MAC address of default gateway for changes
Nice addition. Me personally being the paranoid type, I don't trust the transparency log for monitoring but rather for writing up the root cause analysis. Reason being is that if someone can legally compel Hetzner and/or Linode and/or LetsEncrypt to do or not do something then the same entity can compel the certificate transparency site to ignore something. But you covered what I would do and that is to have multiple nodes doing active monitoring of TLS changes using an external service. That service being openssl s_client in my case.
In the case of Jabber it might be interesting to add some monitoring in the application that uses a cryptographic signed payload to list all the valid fingerprints and send an alert message to the server if something odd is happening like public key pinning but without a hard-fail. That list could be pre-loaded with new certificate fingerprints prior to being deployed. If the oddness is confirmed then perhaps add some way to tell the clients that certificate is likely forged. That way both server operator and client are aware that something evil this way comes.
[Edit] - Looks like based on the comment from MattJ100 that some Jabber servers and some clients can already do something like this.
> if someone can legally compel Hetzner and/or Linode and/or LetsEncrypt to do or not do something then the same entity can compel the certificate transparency site to ignore something
the major certificate transparency logs are operated by several independent global companies, Apple and Google for LE. it's unlikely that they will agree to forge their global CT logs for a single government. more importantly, SCTs allow cryptographic proof of anyone lying, making such an action very dangerous for their continued participation in the WebPKI ecosystem.
5 replies →
The problem is that Let's Encrypt issued a fake certificate without proper validation.
UPD: I was initially wrong. It looks like many CAs will issue a valid certificate to attacker capable of doing MitM.
2 replies →
I mentioned this in the operators channel just now, but also worth noting is that channel binding (e.g. SCRAM-*-PLUS in combination with RFC 9266) also mitigates this attack. Essentially the connecting client is able to detect the mismatch between the certificate or handshake the server thinks it is presenting, and the one the client actually sees.
ejabberd and Prosody support it, and a number of clients too.
> I mentioned this in the operators channel just now, but also worth noting is that channel binding (e.g. SCRAM-*-PLUS in combination with RFC 9266) also mitigates this attack.
Doesn't that require the server to persistently store user passwords in plaintext?
2 replies →
DANE (with DNSSEC) may be an option to look at too, with public keys in DNS. though probably client support isn't universal (i have no (recent) experience using jabber). the CA was one weak point here.
checking for new public keys in certificates when keeping private keys between renewals would also help.
i suppose any storage and possibly ram should also be considered compromised? e.g. private keys stored on disk without encryption, but also by being read into memory. especially on the vm.
DNSSEC operators can be strong-armed the same way. You will also lose out on transparency logs.
In the end law enforcement can also walk up to the machine/hypervisor and steal/monitor interesting things from that as well. This is funnily the exact evil maid threat scenario so many (especially Linux people) find unrealistic.
5 replies →
You don’t need DANE to thwart this particular attack: DNSSEC and CAA could have prevented the MITM certificates from being issued in the way they were. If the xmpp.ru and jabber.ru domains used DNSSEC and CAA, it would not have been enough for the MITM to strong-arm Linode and Hetzner: they would also have to strong-arm the domains’ DNS providers. I don’t think the attackers could have done it stealthily (unnoticed for 3 months) without the co-operation of the .ru registry and a MITM attack on all of the Let’s Encrypt validation vantage points.
I am one of the co-authors of the DANE SRV RFC https://www.rfc-editor.org/rfc/rfc7673 which is what XMPP would use. I don’t follow XMPP development so I don’t know if it has been deployed. I would like it if DANE were more widely used, but it’s not pertinent to this attack.
I think it's a great example why DNSSEC would be bad: at least here, we had transparency logs and there was a simple method to get an attack notification. With DNSSEC, there would be no such thing.
4 replies →
There's also a DNS CAA record that sets up a whitelist of authorities that can issue certificates for the domain.
At least for Let’s Encrypt, you can also use CAA to require a specific account as well or validation method. This could prevent your hosting provider from issuing on your behalf if they don’t control DNS.
https://letsencrypt.org/docs/caa/#the-accounturi-parameter
I use those as well. I am curious if there has ever been a case where LetsEncrypt were legally compelled to create exemptions for domains as in ignoring CAA and not logging to the transparency log or to just outright issue certs to an agency for specific names. CAA account and method restrictions would be negated at that point.
The more I think about it I would wager that Linode and Hetzner were just law enforcement having good taste in VPS providers. It's more likely to me that LE was the compelled target.
5 replies →
Let's encrypt (and other CAs) should not allow validation using unsecure schemes like HTTP. Especially for sites that have valid certificate from other CA.