Comment by yorwba

It's indeed a big advance for interpretability, but that alone doesn't solve alignment. It doesn't even solve interpretability, as there is still a large number of unexplained neurons, just fewer than with earlier approaches.

Once interpretability is solved and we have a fully human-readable accurate explanation of how a model produces its output, that still leaves the problem of spotting problems ahead of time. E.g. for the hummingbird misclassified as ice cream https://already-clean.goodfire.com/assets/september-revision... the method would have to flag that the texture of ice crystals and of feathers, and the shape of a spoon handle and a bird's beak are insufficiently separated. Purely based on the explanation, without having to rely on trial-and-error to find corner cases like this.

And even when it becomes possible to automatically spot issues, that doesn't mean they will all be solvable. Maybe there are tradeoffs where fixing one thing breaks another, and which choice is the better one depends on information you don't have.