Comment by mmh0000
6 days ago
LLMs are good at producing what they/the public know.
In this case:
LLMs know the USB Spec very well.
LLMs know how to read raw packet dumps.
LLMs know how to convert a packet dump to USB spec
LLMs know how to write code to generate USB packets from the spec.
LLMs are also VERY good at transliteration, i.e., converting known-good Python to Rust.
Basically, If you have a well-documented problem, the LLM is a shortcut to learning it yourself. LLMs fail when you have a novel or poorly documented problem. They also fail when you provide the LLM with terrible context or too much context.
Don't sell in-context learning short. Right now I'm waiting on Claude to wrap up the latest of a half-dozen extensive changes to XML files for a fairly-obscure (and obsolete) closed-source electronics CAD program. I am pretty sure it doesn't know anything about these files besides what's in the XML .DTD file (which I also gave it.)
This is a very novel, reasonably-poorly-documented problem, and so far it has batted 1.000.
Kinda sounds like you are finally realizing the dream of XML and DTD envisioned 25 years ago.
Exactly! Not to mention the belated realization of the so-called "expert systems" we were promised back in the 80s. Better late than never.
1 reply →