Comment by michael0church
3 hours ago
This is a great point. The complexity sources are entirely different. An OS is complex because it’s solving a difficult problem and we’ve seen decades of evolution both in knowledge and-how and, at the frontier, in performance standards.
The abilities of LLMs are emergent. You can experiment with LLMs and know as much about their observable behavior as the experts. But there’s no way to “crack open” an LLM and see precisely where each skill or tendency lives; as far as we currently understand, it’s all mashed together.
Learning how to build emergent systems is also a skill kids should learn these days. The closest I got was coding up game of life for CSE 142 (intro programming).
If stochastic gradient descent isn't taught in whatever CS Theory 101 is now, it really should be these days.
> there’s no way to “crack open” an LLM and see precisely where each skill or tendency lives
Mechanistic Interpretability has entered the chat.
For a classic example, see https://www.anthropic.com/research/tracing-thoughts-language...
The spirit of your point stands, though. This kind of research is interesting to read about, but it's very hard, more like neuroscience or biology than computer science ("LLMs are grown, not made"). You're dealing with a lot of extremely _messy_ complexity, for which organic life is really the only good point of comparison. Most of us here are't really equipped for that kind of work; it's not at all like, say, reverse-engineering a piece of software written by humans. And of course the only people who can do it on frontier models from Anthropic and OpenAI are people within the labs themselves. (But I'm optimistic we'll see more of this work on open weights models...)