Method2025-07-30#interpretability
Activation patching to find the cause
Probes tell you what is represented. Patching tells you what matters. By swapping one activation and watching the effect, you move from 'this information is present' to 'this component causes the behavior'.
Sweep the patch across layers and positions to localize the circuit. The spots where patching flips the answer are the load-bearing ones; everything else is along for the ride.
The closest thing in this field to a controlled experiment.
Aha moment
Correlation says a signal is present. Patching asks whether changing that signal changes the outcome.
Try this
Patch one layer-position activation from a clean run into a corrupted run and track whether the answer flips.
Watch for
- Large sweeps without a clean causal hypothesis
- Output flips caused by formatting rather than the target behavior
- A single patch interpreted without replication