Reminds me of the "Robin Hood and Friar Tuck" hack.
Excerpts:
> They dug around in the operating-system listings and devised a thoroughly devilish set of patches. These patches were then incorporated into a pair of programs called 'Robin Hood' and 'Friar Tuck'. Robin Hood and Friar Tuck were designed to run as 'ghost jobs' (daemons, in Unix terminology); they would use the existing loophole to subvert system security, install the necessary patches, and then keep an eye on one another's statuses in order to keep the system operator (in effect, the superuser) from aborting them.
> They found the bandit ghost jobs running, and killed them... and were once again surprised. When Robin Hood was gunned, the following sequence of events took place:
id1: Friar Tuck... I am under attack! Pray save me!
id1: Off (aborted)
id2: Fear not, friend Robin! I shall rout the Sheriff of Nottingham's men!
id1: Thank you, my good fellow!
> Each ghost-job would detect the fact that the other had been killed, and would start a new copy of the recently slain program within a few milliseconds. The only way to kill both ghosts was to kill them simultaneously (very difficult) or to deliberately crash the system.
> Finally, the system programmers did the latter - only to find that the bandits appeared once again when the system rebooted!
How does Robin (id1) manage to print the "I am under attack" message? Shouldn't a kill be immediate?
Couldn't they just do something like sending a SIGSTOP to processes, or use killall in a tight loop? I get it's not Unix, but it seems reasonable similar concepts should exist.
Also, this is more in line with Ken Thompson's "Reflections on Trusting Trust"
That's what I was thinking while reading this article, +1!
In short, the premise of the talk is: if you add a self replicating change to any self-hosted compiler:
- You can 'teach' the compiler to do things that are never explicitly stated in their source code, like, in this article, making the compiler modify programs that meet specific predicates.
- You can not trust that your compiler hasn't been modified without bootstrapping it yourself. You can't trust a program without trusting its compiler; and since compilers are programs, well it's turtles all the way down.
To build a trusted compiler, you have to bootstrap one yourself from some trusted manner, usually from hand-written assembly. In C, this would be akin to writing a small C compiler in assembly, using that to compile tcc, using that to compile a slightly larger C compiler, and that to compile clang or gcc.
But in other more complex languages, such as Rust, bootstrapping is not as clear cut. There's the Reproducible Builds project, but afaict there exists no minimal small Rust compiler you can bootstrap the latest one from: You have to go back the the original OCaml one and compile up from there, or just trust that the publicly available build is secure (which it most likely is).
I guess the lesson is that it's possible to inject hidden self-replicating behavior into any significantly complex system.
An assembler can also do this same attack. You have to hand-assemble your basic assembler to machine code. In theory a sophisticated-enough hex editor could also be subverted, silently adding in code when it's used to edit executable-looking files. The system dynamic linker can be compromised to subvert any executable. Etc.
If a machine is compromised, it can't be recovered without access to a known-good machine.
I get the theory here, but never having written a compiler myself, I’m curious how easy/difficult it would be to come up with those predicates in practice?
Holy crap. That went way deeper than I expected it to.
About halfway through, I was ready to copy the source to another computer and compile it there instead, and then suggest they nuke the original computer.
Turns out I was right. I just didn't know how right I was, which could be important for whatever they plan to do to the person who hacked that computer.
Given the computer model involved, this is probably in the late 80s. If you've read The Cuckoo's Egg, it suggests that getting law enforcement to take computer intrusion seriously was very hard back then.
I suppose the university could impose some kind of sanctions such as revoking the student's degree.
>I was hired by a psychologist to fix a program that seemed to have "strange output" written by one of his ex-grad students.
>printed some white supremacy message
So I'm missing some details here maybe, but isn't that easier to ask the ex-student? He's probably responsible for all of this right? I don't see how anyone could get away with such thing.
That seems unlikely. From my reading there was a program (a simple research tool to ask people psychology questions), that was written by an "ex-Grad Student". Presumably it worked fine for some time, and then started exhibiting strange behaviour (after all if it was written to do this from the get go who would ever have used it?).
If we are going to do detective guess-work (#) get the list of students who took the test. This level of effort to poison a research lab suggests to me the questions raised some uncomfortable issues with one of the respondents, who happened to have compiler abilities and the ability to read the root password on the post-it note on professors desk.
(#) which frankly is how most detectives dive in - that one seems suspicious, look for evidence.
Considering this was malicious sabotage, I doubt the ex-student was even in contact with his former advisor, and regardless a conversation probably wouldn't be very helpful.
Every time I read these stories I wonder if they arent, just that, stories, then I realize it would be kinda fun to create one of these lowlevel compiler infecting things, and realize nah there are atleast a few out there written for shits and giggles. If nothing else, just because of the stories.
The best way to do a modern rendition would be to couple it with some sort of infrastructure vulnerability like a package-squatting attack or similar, so it has a chance to infiltrate for at least a little while before getting killed.
The point would be to have it simply persist - and ideally for a little while too - but do nothing useful other than for example allocate an additional string on a local stack somewhere or something.
Absolute best case scenario would be to announce its existence one year after successful infiltration. :D
> Several days later, an AT&T tech shows up with a disk and loads the proper compile and linker source and we recompile the compiler from the source.
Because he would bill for less time if he just showed up with the compiler binaries and copied them over the rogue ones?
"Let's take pristine source across the air-gap, and manipulate it on the infected computer where there is nonzero risk that the wrong compiler could somehow compile it."
What if the kernel, C library or utilities in /bin and /usr/bin had been involved too?
This level of dedication to white supremacy is unsurprising to me after spending years on the internet, yet there are so many people who refuse to believe it's a real problem.
It is easy to mistake 14-16 year olds with a lot of time on their hands for dedicated real supremacists. The best example was hordes of teenagers teaching a Microsoft chat bot to be a supremacist. Clearly they were having fun.
It's their way of rebelling against the establishment by trolling. Since a teenager can no longer troll by assuming a leftist position, they must do so by pretending to be far right.
I don't think teenagers being racist is a normal activity that is whimsical fun. There are actual consequences to people. Do you think the black 12 to 16 year old with an interest in technology continues that interest when the chat bot starts yelling racial slurs at him and telling him his life his worthless. It's very easy for people who have never experienced racism to say he should just brush it off. "It was just a joke bro" is also one of the favorite tools of neo nazis and other groups to gauge interest from other people while being able to claim they aren't actually racist.
The story is also about a former grad student so he is well into adult hood but probably started off as one of those teenagers just "trolling".
14-16 year olds who are actively participating in white supremacist movements are white supremacists.
The person being targeted by their harassment campaign can't know and doesn't care that they're minors or they think they're kidding or whatever.
The harm is the same and the right-wing radicalization of young white men online is an extremely serious problem. People in this age range have killed people for this idealogy!
Reminds me of the "Robin Hood and Friar Tuck" hack.
Excerpts:
> They dug around in the operating-system listings and devised a thoroughly devilish set of patches. These patches were then incorporated into a pair of programs called 'Robin Hood' and 'Friar Tuck'. Robin Hood and Friar Tuck were designed to run as 'ghost jobs' (daemons, in Unix terminology); they would use the existing loophole to subvert system security, install the necessary patches, and then keep an eye on one another's statuses in order to keep the system operator (in effect, the superuser) from aborting them.
> They found the bandit ghost jobs running, and killed them... and were once again surprised. When Robin Hood was gunned, the following sequence of events took place:
> Each ghost-job would detect the fact that the other had been killed, and would start a new copy of the recently slain program within a few milliseconds. The only way to kill both ghosts was to kill them simultaneously (very difficult) or to deliberately crash the system.
> Finally, the system programmers did the latter - only to find that the bandits appeared once again when the system rebooted!
Quoted in the Jargon File: http://www.catb.org/jargon/html/meaning-of-hack.html
Original story: https://www.cs.utah.edu/~elb/folklore/xerox.txt
A couple things I wonder there:
How does Robin (id1) manage to print the "I am under attack" message? Shouldn't a kill be immediate?
Couldn't they just do something like sending a SIGSTOP to processes, or use killall in a tight loop? I get it's not Unix, but it seems reasonable similar concepts should exist.
Also, this is more in line with Ken Thompson's "Reflections on Trusting Trust"
You can put a signal handler in your program to catch the SIGSTOP and run some code before it gets killed.
5 replies →
I almost feel like that's too commonplace to be noteworthy anymore? There are lots of annoying programs I try to kill only for them to be restarted.
This is modelled on "Reflections on Trusting Trust" by Ken Thompson, 1984
https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_Ref...
That's what I was thinking while reading this article, +1!
In short, the premise of the talk is: if you add a self replicating change to any self-hosted compiler:
- You can 'teach' the compiler to do things that are never explicitly stated in their source code, like, in this article, making the compiler modify programs that meet specific predicates.
- You can not trust that your compiler hasn't been modified without bootstrapping it yourself. You can't trust a program without trusting its compiler; and since compilers are programs, well it's turtles all the way down.
To build a trusted compiler, you have to bootstrap one yourself from some trusted manner, usually from hand-written assembly. In C, this would be akin to writing a small C compiler in assembly, using that to compile tcc, using that to compile a slightly larger C compiler, and that to compile clang or gcc.
But in other more complex languages, such as Rust, bootstrapping is not as clear cut. There's the Reproducible Builds project, but afaict there exists no minimal small Rust compiler you can bootstrap the latest one from: You have to go back the the original OCaml one and compile up from there, or just trust that the publicly available build is secure (which it most likely is).
I guess the lesson is that it's possible to inject hidden self-replicating behavior into any significantly complex system.
You can use mrustc to not have to go back to the original OCaml.
2 replies →
An assembler can also do this same attack. You have to hand-assemble your basic assembler to machine code. In theory a sophisticated-enough hex editor could also be subverted, silently adding in code when it's used to edit executable-looking files. The system dynamic linker can be compromised to subvert any executable. Etc.
If a machine is compromised, it can't be recovered without access to a known-good machine.
https://github.com/certik/bcompiler/ is an effort to do this very bootstrapping.
I get the theory here, but never having written a compiler myself, I’m curious how easy/difficult it would be to come up with those predicates in practice?
1 reply →
Coincidentally, Computerphile just posted this today:
https://www.youtube.com/watch?v=SJ7lOus1FzQ
Holy crap. That went way deeper than I expected it to.
About halfway through, I was ready to copy the source to another computer and compile it there instead, and then suggest they nuke the original computer.
Turns out I was right. I just didn't know how right I was, which could be important for whatever they plan to do to the person who hacked that computer.
Given the computer model involved, this is probably in the late 80s. If you've read The Cuckoo's Egg, it suggests that getting law enforcement to take computer intrusion seriously was very hard back then.
I suppose the university could impose some kind of sanctions such as revoking the student's degree.
>I was hired by a psychologist to fix a program that seemed to have "strange output" written by one of his ex-grad students.
>printed some white supremacy message
So I'm missing some details here maybe, but isn't that easier to ask the ex-student? He's probably responsible for all of this right? I don't see how anyone could get away with such thing.
That seems unlikely. From my reading there was a program (a simple research tool to ask people psychology questions), that was written by an "ex-Grad Student". Presumably it worked fine for some time, and then started exhibiting strange behaviour (after all if it was written to do this from the get go who would ever have used it?).
If we are going to do detective guess-work (#) get the list of students who took the test. This level of effort to poison a research lab suggests to me the questions raised some uncomfortable issues with one of the respondents, who happened to have compiler abilities and the ability to read the root password on the post-it note on professors desk.
(#) which frankly is how most detectives dive in - that one seems suspicious, look for evidence.
Considering this was malicious sabotage, I doubt the ex-student was even in contact with his former advisor, and regardless a conversation probably wouldn't be very helpful.
Every time I read these stories I wonder if they arent, just that, stories, then I realize it would be kinda fun to create one of these lowlevel compiler infecting things, and realize nah there are atleast a few out there written for shits and giggles. If nothing else, just because of the stories.
The best way to do a modern rendition would be to couple it with some sort of infrastructure vulnerability like a package-squatting attack or similar, so it has a chance to infiltrate for at least a little while before getting killed.
The point would be to have it simply persist - and ideally for a little while too - but do nothing useful other than for example allocate an additional string on a local stack somewhere or something.
Absolute best case scenario would be to announce its existence one year after successful infiltration. :D
Did this sort of read like a writing-prompt? By day 4 he could have unraveled a larger ploy involving lizard people.
Putting the ‘Code’ into DaVinci Code.
> Several days later, an AT&T tech shows up with a disk and loads the proper compile and linker source and we recompile the compiler from the source.
Because he would bill for less time if he just showed up with the compiler binaries and copied them over the rogue ones?
"Let's take pristine source across the air-gap, and manipulate it on the infected computer where there is nonzero risk that the wrong compiler could somehow compile it."
What if the kernel, C library or utilities in /bin and /usr/bin had been involved too?
So tl;dr
- the original creator created a self-replicating virus that would modify this one program, making it unmodifiable.
- actually it was in the compiler. woop. can't fix it without external fixes
- creates a backdoor into the system, could have fucked with the whole campus' data
- he would have gotten away with it too, if not for the fact that he made this supremacist text
So basically, dang.
This level of dedication to white supremacy is unsurprising to me after spending years on the internet, yet there are so many people who refuse to believe it's a real problem.
It is easy to mistake 14-16 year olds with a lot of time on their hands for dedicated real supremacists. The best example was hordes of teenagers teaching a Microsoft chat bot to be a supremacist. Clearly they were having fun.
It's their way of rebelling against the establishment by trolling. Since a teenager can no longer troll by assuming a leftist position, they must do so by pretending to be far right.
I don't think teenagers being racist is a normal activity that is whimsical fun. There are actual consequences to people. Do you think the black 12 to 16 year old with an interest in technology continues that interest when the chat bot starts yelling racial slurs at him and telling him his life his worthless. It's very easy for people who have never experienced racism to say he should just brush it off. "It was just a joke bro" is also one of the favorite tools of neo nazis and other groups to gauge interest from other people while being able to claim they aren't actually racist.
The story is also about a former grad student so he is well into adult hood but probably started off as one of those teenagers just "trolling".
13 replies →
Give 'em 5-10 years, and it's a lot harder to tell them apart.
they exist offline too. it's a very disturbing trend.
14-16 year olds who are actively participating in white supremacist movements are white supremacists.
The person being targeted by their harassment campaign can't know and doesn't care that they're minors or they think they're kidding or whatever.
The harm is the same and the right-wing radicalization of young white men online is an extremely serious problem. People in this age range have killed people for this idealogy!