Method2025-09-14#interpretability

Sparse autoencoders to untangle neurons

The trick is over-completeness plus sparsity. You map activations into a much wider space but force only a few of those dimensions to be active at once. The pressure pushes tangled concepts apart into separate, more interpretable features.

Once you have the features you can do surgery: amplify one and watch the model fixate on a bridge, suppress another and watch a behavior fade. It is the closest thing we have to named knobs inside the network.

This is roughly how the 'Golden Gate Claude' demo worked.

Aha moment

Sparse features give you named-ish knobs because they separate mixed neuron behavior into cleaner directions.

Try this

Treat an SAE feature as a hypothesis, then test whether steering it changes the behavior you care about.

Open NotesEntry JSON

Watch for

  • Human-readable labels that overfit a few examples
  • Feature steering without downstream behavior checks
  • Polysemantic residue hiding inside a cleaner-looking feature