Tool2025-10-19#tooling
BertViz for watching attention move
BertViz draws the attention pattern as lines connecting tokens, split by head and layer. It runs in a notebook in a couple of lines and works on most Hugging Face transformer models.
The value is not the pretty picture, it is the habit of looking. Once you have seen a head that exclusively tracks matching brackets, you stop thinking of the model as a black box and start thinking of it as a stack of small, legible mechanisms.
Most heads are boring. The interesting 5% are worth the dig.
Aha moment
Attention is not the whole mind, but it is a useful map of what information is being routed where.
Try this
Inspect heads for repeated roles: bracket matching, quote tracking, subject agreement, or copied labels.
Watch for
- Pretty attention maps with no behavioral test
- One dramatic head treated as the whole explanation
- Patterns that disappear after small prompt perturbations