Comment by ericbarrett
5 years ago
One of the most fascinating breach analyses I've ever read.
Reading between the lines, I sense the client didn't 100% trust Mr. Bogdanov in the beginning, and certainly knew there was exfiltration of some kind. Perhaps they had done a quick check of the same stats they guided the author toward. "Check for extra bits" seems like a great place to start if you don't know exactly what you're looking for.
Their front-end architecture seemed quite locked down and security-conscious: just a kernel + Go binary running as init, plain ol' NFS for config files, firewalls everywhere, bastion hosts for internal networks, etc. So already the client must have suspected the attack was of significant sophistication. Who was better equipped to do this than their brilliant annual security consultant?
Which is completely understandable to me, as this hack is already of such unbelievable sophistication that resembles a Neil Stephenson plot. Since the author did not actually commit the crime, and in fact is a brilliant security researcher, everything worked out.
> So already the client must have suspected the attack was of significant sophistication. Who was better equipped to do this than their brilliant annual security consultant?
If you suspected your security consultant, what would be the point of slipping them tiny hints about what you've found? If they're the source of the intrusion, they already know. If they're not the source of the intrusion, why fear them when you've already been compromised? Also, if you suspected the consultant, why hire them to do the security review?
I suspect the real reason is probably simpler: they have strong personal or financial incentives to "not have known" about the intrusion before the researcher discovered it.
I agree there's nothing to rule out your theory. Likely we will never know. But then why authorize sharing the story?
Specifically I don't think the owner thought it was likely, just a concern he couldn't shake. Probably he relaxed as soon as the consultant didn't make excuses, and tackled the job—extracting the binary from an unlinked inode is definitely not showing reluctance. Pure speculation, of course.
Hi, author of the blog post. This is correct - keeping PII protected has always been their concern, but recent breaches in thier's and other industries (including some they heard of and were not publicized) made them even more concerned.
I don't know if this is realistic in any way, but I've seen lots of Murder, She Wrote episodes where the criminal only gets caught because they become involved in the investigation some way and accidentally reveal knowledge that only the attacker could possibly know. This strategy necessitates hiding secret information so it can be revealed later by the attacker.
> just a kernel + Go binary running as init
This is hardly reducing the attack surface compared to a good distro with the usual userspace.
It's been decades since attackers relied on a shell, or unix tools in general, or on being to write to disk and so on: it's risky and ineffective.
Many attack tools run arbitrary code inside the same process that has been breached and extract data from its memory.
They don't try to snoop around or write to disk and so on. Rather, move to another host.
The only good mitigation is to split your own application in multiple processes based on the type of risk and sandbox each of them accordingly.
> This is hardly reducing the attack surface compared to a good distro with the usual userspace.
Run `tcpdump -n 'tcp and port 80'` on your frontend host and you'll still see PHP exploit attempts from 15 years ago. Not every ghost who knocks is an APT. A singleton Go binary running on a Linux kernel with no local storage is objectively a smaller attack surface than a service running in a container with /bin/sh, running on a vhost with a full OS, running on a physical host with thousands of sleeping VMs—the state of many, many websites and APIs today.
> is objectively a smaller attack surface
No, you have to understand what is really part of the attack surface and what the attacker wants.
For example, on a properly built system with a single application running with its own user the attacker might have no practical benefit at all in doing a privilege escalation to root.
> running on a physical host with thousands of sleeping VMs
This is a strawman. A shared hypervisor opens another attack surface and was not part of the discussion.
4 replies →