Comment by rikafurude21
19 hours ago
Its still crazy to me that everyone has a pocket AI-hacker ready to inspect firmware and modify their devices now. You just put the agent on it and it gives you access in minutes. You would have to be a Hotz tier hacker if you wanted to do anything close to this only last year, or at the very least extremely patient for long hours.
> You would have to be a Hotz tier hacker if you wanted to do anything close to this only last year
This isn't true at all. Yes, LLMs have made it dramatically easier to analyse, debug and circumvent. Both for people who didn't have the skill to do this, and for people who know how to but just cannot be bothered because it's often a grind. This specific device turned out to be barely protected against anything. No encrypted firmware, no signature checking, and built-in SSH access. This would be extremely doable for any medium skilled person without an LLM with good motivation and effort.
You're referring to George Hotz, which is known for releasing the first PS3 hypervisor exploit. The PS3 was / is fully secured against attackers, of which the mere existence of a hypervisor layer is proof of. Producing an exploit required voltage glitching on physical hardware using an FPGA [1]. Perhaps an LLM can assist with mounting such an attack, but as there's no complete feedback loop, it still would require a lot of human effort.
[1] https://rdist.root.org/2010/01/27/how-the-ps3-hypervisor-was...
The hacking aspect has been hit and miss for me. Just today I was trying to verify a fix for a CVE and even giving the agent the CVE description + details on how to exploit it and the code that fixed it, it couldn't write the exploit code correctly.
Not to say it's not super useful, as we can see in the article
CVEs and all, but I just can't wait for firmwares for cheaper modern cameras from Sony, Nikon and Panasonic getting hacked and modified too add features from more expensive models.
They're all firmware restricted to justify buying more expensive models, in one way or another way.
DNG support would be pretty awesome too.
>... but as there's no complete feedback loop, it still would require a lot of human effort.
Not for long. Picture this: a robot receives instructions on what to physically solder in order to complete the desired modification task.
However, before it can send an image back to the vision-aware LLM guiding it, the PCB lights on fire along with the robot because said LLM confidently gave the wrong instructions.
Then, the robotic fire brigade shows up and mostly walks into walls unable to navigate anywhere useful.
The future is bright.
Minor correction. At 27c3's "Console Hacking 2010" talk. Geohot's Hypervisor work is mentioned at 4:25 or so. Described as "really unreliable" and "eh whatever" due to requiring hardware modification and only granting rudimentary hypervisor access.
These were the same people that then went on to explain how they reverse-engineered the encryption keys of the PS3 to enable "fakesigned" code to be installed
didn't PS3 have a hardcoded nonce for their ECDSA impl that allowed full key recovery? I would agree that I doubt LLMs let people mount side-channel attacks easily on consumer electronics though.
Yes indeed, that chain of exploits was all software and not hardware. Developed after the Hotz exploit and Sony subsequently shuttering OtherOS.
It didn't directly give access to anything however. IIRC they heavily relied on other complex exploits they developed themselves, as well as relying on earlier exploits they could access by rolling back the firmware by indeed abusing the ECDSA implementation. At least, that turned out to be the path of least resistance. Without earlier exploits, there would be less known about the system to work with.
Their presentation [1] [2] is still a very interesting watch.
[1] https://www.youtube.com/watch?v=5E0DkoQjCmI
[2] https://fahrplan.events.ccc.de/congress/2010/Fahrplan/attach...
1 reply →
> fully secured against attackers, of which the mere existence of a hypervisor layer is proof of
https://en.wikipedia.org/wiki/Virtual_machine_escape
The last one was 8 years ago. It's not a terribly common vuln anymore - not that it ever was.
1 reply →
> Perhaps an LLM can assist with mounting such an attack, but as there's no complete feedback loop, it still would require a lot of human effort.
LLMs have had no problem modifying software on an attached android phone. It's only a matter of time.
From the article, it sounds like he used Claude Code as an alternative to Wireshark and Google to decode USB HID traffic and find protocol documentation, respectively.
I suppose this could save a bit of time if you don't already have Wireshark installed, with a minor risk of hallucinations.
Other than this, he used Docker for some reason* to edit ~root/.ssh/authorized_keys and /etc/shadow in the firmware tarball, then wrote a quick Python script to send the relevant HID messages and copy the modified tarball to a volume mounted from a USB drive exposed by the device in response to one of the HID messages.
Maybe he used Claude to do some of this other stuff. Who knows? But the only thing in the post or the linked scripts that wasn't immediately obvious to me is why he installed the whois package in his Ubuntu container, but it turns out that, in Debian, the mkpasswd utility is installed by the whois package for historical reasons[1].
So basically, you have to be an insane hacker, or else have a basic working knowledge of Linux system administration (or at least know how to use the man(1) command; then again Google would probably suffice as an alternative) and how to write trivial programs in any language with bindings to a USB HID library.
* Presumably because he was on a Mac and didn't have a Linux box handy to generate the hashed password (which requires using glibc crypt(3) in a way that isn't compatible with macOS libc crypt(3), so nontrivial on a Mac).
Not sure why he needed password authentication in the first place, but, at the author's request, I won't shoot him.
I will, however, point out that, unless the sshd_config file on the device already set PermitRootLogin to something other than the default "prohibit-password", password authentication wouldn't have worked to log in as root, even with PasswordAuthentication set to "yes".
[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=116260
I used wireshark to capture the traffic, and looked thru the pcap for the area that looked like the updating, and gave the packet numbers and the pcap to claude code to find the details of how it worked instead of scribbling notes for an hour or two i’d guess
I’m very used to doing this stuff manually for various devices and software, but am also interested in tracking llm progress, and it seemed simple enough to get a rundown of what was happening while I did other work.
It was the first time I have messed around with hid devices though, so that was aided by claude
and yeah i’ve been bit by having to google how to get mkpasswd dozens of times over the years and used to have to do a lot of rootfs editing on a mac, so I got used to doing it in a container.
no real reason for wanting pw auth, I ended up turning it off afterwards but it’s been a bit since I wrote this
thanks for the comment!
> a bit of time
A bit of time is an understatement.
I used Wireshark to analyze various things (mostly smart home) over the years, but now CC does in minutes what it would take me a few hours before - and provides dedicated, custom made panels for whatever I want.
As an example - debugging KNX magistrale in my home, previously it was either wireshark and a ton of regexes, handwritten scripts (or official software that was terrible), now you just tell CC what you want to extract, and you get beautiful real-time views of the activity.
One thing is previewing the traffic, but then CC can easily fetch docs for any device it finds on the network, if it has an API (official or not), utilize it and do whatever you want.
This 1000% - I’ve used AI to enable SSH in one Phase One digital back I own, and to reverse engineer and patch the firmware on another to make the back think it’s a different back - Credo 50 to IQ250! The internals are literally the Sam.
I'm sorry, are you trusting an LLM to touch a camera that costs like a new car?
Only a little bit of touching for the really expensive one. The Credo 50 was less than 1K though.
Also Phase One Support/Repair is absolutely phenomenal and unless you toast the sensor; repairs are “fairly” economical.
[dead]
its really nice to not have to spend hours looking thru packet captures and stuff, i enjoy digging but as i'm getting older I have less time to spend 16 hour days looking at random firmware blobs
LLM are not capable of doing that for most things. Having an open ssh device does not require any special "skill".
If it’s embedded Linux with no HAB it’s not hard to make “adjustments.” Just use file and binwalk to figure out what it is and break it open.
Damn, maybe I can throw an agent at trying to unlock IMEI spoofing on my Unifi LTE modem. That one guy on twitter who does all the LTE modem unlocking never replied to my tweet :(
there’s barely any hacking here
the guy found this through looking at the firmware but nmap -p 22 would have also found this
So like the first thing you would do to attack the device
I found an issue exactly like this on an ISP-provided router. I am nowhere near geohot but also didn’t even do as much as the guy in the article lmao
to me this is just normal to do with your devices. I think it’s interesting because it has no fw signing etc and because they left ssh, not because of figuring out how to do the patching.