Comment by AndrewKemendo
19 days ago
No it’s not
Nobody has built a human so we don’t know how they work
We know exactly how LLM technology works
19 days ago
No it’s not
Nobody has built a human so we don’t know how they work
We know exactly how LLM technology works
We know _how_ it works but even Anthropic routinely does research on its own models and gets surprised
> We were often surprised by what we saw in the model
https://www.anthropic.com/research/tracing-thoughts-language...
Which is…true of all technologies since forever
Except it's not. Traditional algorithms are well understood because they're deterministic formulas. We know what the output is if we know the input. The surprises that happen with traditional algorithms are when they're applied in non-traditional scenarios as an experiment.
Whereas with LLMs, we get surprised even when using them in an expected way. This is why so much research happens investigating how these models work even after they've been released to the public. And it's also why prompt engineering can feel like black magic.
8 replies →
We know why they work, but not how. SotA models are an empirical goldmine, we are learning a lot about how information and intelligence organize themselves under various constraints. This is why there are new papers published every single day which further explore the capabilities and inner-workings of these models.
You can look at the weights and traces all you like with telemetry and tracing
If you don’t own the model then you have a problem that has nothing to do with technology
Ok, but the art and science of understanding what we're even looking at is actively being developed. What I said stands, we are still learning the how. Things like circuits, dependencies, grokking, etc.
[dead]