Comment by JimSanchez

Fascinating idea that LLM performance might improve simply by changing the inference path through existing layers rather than retraining weights. It’s interesting to think of transformer stacks developing something like functional “circuits” similar to brain regions.