Comment by mike_hearn

3 days ago

The essay says he has extended it to LLMs, although not as much.

The connection to alignment seems pretty clear. Steering DNNs is well understood, the bottleneck is identifying what to steer. Once you can easily identify neural circuits then turning them up/down to achieve specific results is straightforward.

Note that this is already being done even without the kind of precision that Nick's work says is possible. Morality abliterations of open weight models are easy to find on HuggingFace. They compute the steering vectors by just presenting the model with lots of disparate examples that have the single moral decision as the commonality, then finding the shared vector directions.