Method2025-09-30#interpretability

Probe for a concept with a linear classifier

The setup is small: collect hidden states from a chosen layer for a labeled dataset, then fit logistic regression on top. High accuracy means the concept is linearly available at that layer.

Sweep the probe across layers and you can watch where a concept first becomes readable. Abstract properties tend to sharpen in the middle layers, which is a clue about where the model is doing the relevant work.

Cheap, fast, and weirdly revealing.

Aha moment

If a tiny linear classifier can read a concept from hidden states, the model has already organized that signal.

Try this

Probe one concept across layers and look for where accuracy first rises above a simple baseline.

Open NotesEntry JSON

Watch for

  • High probe accuracy mistaken for causal importance
  • Labels that leak through surface wording
  • Layer sweeps without a held-out validation split