And with the Linux Kernel being friendly to both LLM usage and a modern language, for some of us who don't know C, a huge barrier is lifted from being able to write device drivers.
Not a successful anecdote, but I have a Windows Hello compatible Kengsington fingerprint reader, and for some time I wanted to write drivers for Linux. Even without using C, it would have been a huge undertaking only to fail in the end; because Claude did much of the research and concluded that the device wouldn't work on Linux (can't remember why but it made sense). Then it suggested what could work.
Back in the day when dial-up modems were a "significant" market, the concept of "WinModems" came around. Minimum hardware, maximum software. If the sensor is good enough to actually work though, it probably won't be long before an agent can implement the driver, if it can't already.
Ugh - thanks for that memory. Worked tech support for Microsoft at that time - we’d try but often the recommendation was to buy a real modem (“But it’s so much more expensive!” - “Yes, but it works….”)
Don't know much about driver maintenance but if it is working why remove it?
What I read in the link was that it was removed because LLM was making suggestions to improve the code/bring it up to modern standards but since it was rarely used no-one wanted to review it not that it was buggy.
>In recent months we have seen a lot of old kernel code removed stemming from that AI/LLM noise.
This is the opposite of what my parent was hoping to see, extended support for old/obscure hardware because we have LLM.
LLMs are good at producing what they/the public know.
In this case:
LLMs know the USB Spec very well.
LLMs know how to read raw packet dumps.
LLMs know how to convert a packet dump to USB spec
LLMs know how to write code to generate USB packets from the spec.
LLMs are also VERY good at transliteration, i.e., converting known-good Python to Rust.
Basically, If you have a well-documented problem, the LLM is a shortcut to learning it yourself. LLMs fail when you have a novel or poorly documented problem. They also fail when you provide the LLM with terrible context or too much context.
Don't sell in-context learning short. Right now I'm waiting on Claude to wrap up the latest of a half-dozen extensive changes to XML files for a fairly-obscure (and obsolete) closed-source electronics CAD program. I am pretty sure it doesn't know anything about these files besides what's in the XML .DTD file (which I also gave it.)
This is a very novel, reasonably-poorly-documented problem, and so far it has batted 1.000.
IMO it's because they don't get burnt out by a lack of results.
After 5-6 consecutive approaches fail, I need a reason to think the next one might work out to stay motivated.
Claude will keep burning credits trying new approaches until something sticks. That's a huge advantage in a field where most of the things you try don't go anywhere.
As an analogy, you're not going to cut your lawn by hand at the speed and slow pace that the smallest and weakest lawn mowing robot does. You're not going to cut your lawn with kitchen scissors. But the LLM will, metaphorically, be perfectly happy to cut your lawn with kitchen scissors as long as it has enough time/tokens to keep cranking away at it.
I had a discussion about this with a friend recently. I was using an LLM with a serial console that it wrote to allow us to share a connection to a Z80 computer running CP/M. I was having trouble with the assembler, it wouldn’t assemble in user 1 of a particular disk, so I asked the LLM for help. It debugged ASM.COM and the BDOS, walked through the source compared to memory dumps, basically banging its head on the wall for an hour or more, until it finally figured out and verified that my disk had a bad block. I never would have put in that much effort to find the root cause of the problem.
> Claude will keep burning credits trying new approaches until something sticks. That's a huge advantage in a field where most of the things you try don't go anywhere.
LLMs don't get as mentally exhausted by trying repeated tedious things as a human will. Same general idea as, imagine if you as a human typing into a keyboard had to manually fuzz test software in a pre-LLM era vs. how fuzz testing is actually done.
LLMs seem to be trained to work very well against a goal, especially one it can verify against. I guess because it can easily know if it passed or failed, va other tasks where good/bad output is subjective
I don't think they're any more skilled at it than someone who knows how to reverse engineer stuff... But it is definitely a place where AI is amazing because reverse engineering is usually extremely time consuming and tedious. AI doesn't care about that.
It also has the benefit that it doesn't usually matter too much if it gets minor details wrong. It's definitely one of the areas - like hacking - where it's a) tedious and b) insensitive to mistakes where AI absolutely shines.
My experience is that they're better than me at a lot of the process, so probably worse than someone who's a full time reverse engineer but as good as or better than most. They'll definitely get some small details wrong that would derail the entire thing, so having some skills that are pretty much "This smells wrong" helps a lot, but I think for many scenarios they'll unblock someone who has little RE experience.
It's LLMs' generalist skills. I think reverse engineering is usually hard when you're in an unfamiliar domain. Eg I've never programmed a videogame or windows application but I'm trying to crack one. On the other hand if you know the domain, and know what the programmers' intentions must be in any given block of code, it's often straightforward. There are no unfamiliar domains for LLMs, including it seems proprietary software.
To some degree. But esp for drivers, you still need to know when they go wrong, and steer them right, or your code will either just not work or be an unmaintainable, not-upstreamable mess.
And with the Linux Kernel being friendly to both LLM usage and a modern language, for some of us who don't know C, a huge barrier is lifted from being able to write device drivers.
Not a successful anecdote, but I have a Windows Hello compatible Kengsington fingerprint reader, and for some time I wanted to write drivers for Linux. Even without using C, it would have been a huge undertaking only to fail in the end; because Claude did much of the research and concluded that the device wouldn't work on Linux (can't remember why but it made sense). Then it suggested what could work.
Back in the day when dial-up modems were a "significant" market, the concept of "WinModems" came around. Minimum hardware, maximum software. If the sensor is good enough to actually work though, it probably won't be long before an agent can implement the driver, if it can't already.
Ugh - thanks for that memory. Worked tech support for Microsoft at that time - we’d try but often the recommendation was to buy a real modem (“But it’s so much more expensive!” - “Yes, but it works….”)
yes! but use rust because that’s where Linux is going come hell or high water
Well good news... er... the opposite actually....
>Additional Old Linux Drivers Face Removal Due To Noise From AI/LLM Coding Agents
https://www.phoronix.com/news/Linux-Retiring-Moxa-Driver
They're being removed due to a lack of maintenance. If anyone cared about that hardware and fixed the bugs, surely they would be kept.
Don't know much about driver maintenance but if it is working why remove it?
What I read in the link was that it was removed because LLM was making suggestions to improve the code/bring it up to modern standards but since it was rarely used no-one wanted to review it not that it was buggy.
>In recent months we have seen a lot of old kernel code removed stemming from that AI/LLM noise.
This is the opposite of what my parent was hoping to see, extended support for old/obscure hardware because we have LLM.
1 reply →
I don't understand the technology enough to see why, but LLMs seem unusually skilled at reverse engineering proprietary software.
LLMs are good at producing what they/the public know.
In this case:
LLMs are also VERY good at transliteration, i.e., converting known-good Python to Rust.
Basically, If you have a well-documented problem, the LLM is a shortcut to learning it yourself. LLMs fail when you have a novel or poorly documented problem. They also fail when you provide the LLM with terrible context or too much context.
Don't sell in-context learning short. Right now I'm waiting on Claude to wrap up the latest of a half-dozen extensive changes to XML files for a fairly-obscure (and obsolete) closed-source electronics CAD program. I am pretty sure it doesn't know anything about these files besides what's in the XML .DTD file (which I also gave it.)
This is a very novel, reasonably-poorly-documented problem, and so far it has batted 1.000.
3 replies →
IMO it's because they don't get burnt out by a lack of results.
After 5-6 consecutive approaches fail, I need a reason to think the next one might work out to stay motivated.
Claude will keep burning credits trying new approaches until something sticks. That's a huge advantage in a field where most of the things you try don't go anywhere.
As an analogy, you're not going to cut your lawn by hand at the speed and slow pace that the smallest and weakest lawn mowing robot does. You're not going to cut your lawn with kitchen scissors. But the LLM will, metaphorically, be perfectly happy to cut your lawn with kitchen scissors as long as it has enough time/tokens to keep cranking away at it.
I had a discussion about this with a friend recently. I was using an LLM with a serial console that it wrote to allow us to share a connection to a Z80 computer running CP/M. I was having trouble with the assembler, it wouldn’t assemble in user 1 of a particular disk, so I asked the LLM for help. It debugged ASM.COM and the BDOS, walked through the source compared to memory dumps, basically banging its head on the wall for an hour or more, until it finally figured out and verified that my disk had a bad block. I never would have put in that much effort to find the root cause of the problem.
> Claude will keep burning credits trying new approaches until something sticks. That's a huge advantage in a field where most of the things you try don't go anywhere.
Unless you're on consumption billing.
LLMs don't get as mentally exhausted by trying repeated tedious things as a human will. Same general idea as, imagine if you as a human typing into a keyboard had to manually fuzz test software in a pre-LLM era vs. how fuzz testing is actually done.
That's actually pretty simple.
Reverse engineering isn't so much hard as it is exhausting.
LLMs simply don't care about exhaustion.
LLMs seem to be trained to work very well against a goal, especially one it can verify against. I guess because it can easily know if it passed or failed, va other tasks where good/bad output is subjective
I don't think they're any more skilled at it than someone who knows how to reverse engineer stuff... But it is definitely a place where AI is amazing because reverse engineering is usually extremely time consuming and tedious. AI doesn't care about that.
It also has the benefit that it doesn't usually matter too much if it gets minor details wrong. It's definitely one of the areas - like hacking - where it's a) tedious and b) insensitive to mistakes where AI absolutely shines.
My experience is that they're better than me at a lot of the process, so probably worse than someone who's a full time reverse engineer but as good as or better than most. They'll definitely get some small details wrong that would derail the entire thing, so having some skills that are pretty much "This smells wrong" helps a lot, but I think for many scenarios they'll unblock someone who has little RE experience.
Also verifiable. LLMs shine when they can know when the task is completed correctly. Otherwise they will finish and hand over something that's wrong.
It's LLMs' generalist skills. I think reverse engineering is usually hard when you're in an unfamiliar domain. Eg I've never programmed a videogame or windows application but I'm trying to crack one. On the other hand if you know the domain, and know what the programmers' intentions must be in any given block of code, it's often straightforward. There are no unfamiliar domains for LLMs, including it seems proprietary software.
To some degree. But esp for drivers, you still need to know when they go wrong, and steer them right, or your code will either just not work or be an unmaintainable, not-upstreamable mess.
Maybe wiimotes will finally work on the Apple silicon bluetooth stack as they already do on Windows and Linus.
This might help; haven’t tried it myself. https://github.com/dolphin-emu/WiimotePair