IA-005 / 2026
HeadVis is a scatter plotA graph where each point represents a pair of values, showing the relationship between two sets of numbers on a coordinate plane.A graph of paired numerical values, with one variable on the horizontal axis and the corresponding value of a second variable on the vertical axis, used to reveal relationships or association between the variables.glossary showing attentionHow the model decides which earlier words matter when it is working on the current one.Which positions a model attends to, and how strongly.glossary, about induction-headsParts of the model that spot a pattern earlier in the text and continue it.Attention heads that continue a pattern seen earlier in the context.glossary, polysemanticityOne part of the model doing several unrelated jobs at once.A single component responding to several unrelated concepts.glossary and attention-headsThe parts that decide which earlier words matter for the next one.What an attention head does, on a chosen task or across the full data distribution.glossary.
- ID
- IA-005
- Name
- HeadVis — interactive tool for investigating attention heads
- Artefact type
- Interactive tool
- Visual form
- Scatter plotA graph where each point represents a pair of values, showing the relationship between two sets of numbers on a coordinate plane.A graph of paired numerical values, with one variable on the horizontal axis and the corresponding value of a second variable on the vertical axis, used to reveal relationships or association between the variables.glossary
- Makes visible
- AttentionHow the model decides which earlier words matter when it is working on the current one.Which positions a model attends to, and how strongly.glossary
- Role of the visual
- Instrument
- Runnable by a visitor
- Pre-set examples only
- Date published
- 2026-05-04
- Year
- 2026
- Link status
- Live
- Built by (people)
- R. Luger; Harish Kamath; Doug Finkbeiner; Purvi Goel; Adam Jermyn; Sam Zimmerman; Joshua Batson; Tom Conerly
- Organisation
- Anthropic
- Venue / published in
- Transformer Circuits Thread
- Sector
- Frontier lab
- Country / region
- US
- Open source
- Partial
- Method / technique
- Attention pattern visualisation; per-head distribution metrics (e.g. induction score); PCA of Q/K/O activations; QK and OV feature attributions via a 10M-feature weakly causal crosscoder
- Model(s) studied
- Claude Haiku 3.5; Gemma 3 1B (public demo)
- Access required
- Open weights
- What it visualises
- Attention head behaviour across a full data distribution — scatter plots of heads by metric, token-level attention patterns, PCA clusters, and QK/OV circuit diagrams
- Interaction affordances
- Scatter-select a head by any metric; hover tokens to see attention; browse dataset examples; run custom sequences (server build); rank all heads on a query-key pair
- Pragmatic vs Basic science
- Basic science
- Reverse-eng vs Concept-based
- Reverse-engineering
- Observational vs Interventional
- Both
- Intended audience
- Researchers
- Description (card)
- An interactive tool for interrogating attention heads, released as the research output in its own right. Its central finding is methodological: a head's behaviour on the full data distribution rarely matches what a narrow task suggests.
- Why it matters
- Reframes the tool as the contribution, and supplies the concrete examples the field needs to attack attention decomposition.
- Visual / design notes
- Scatter-plot head selector with configurable axes is the smart move — it makes 'find an interesting head' a two-click operation instead of a research project.
- Tags
- modality:languageText. Models that read and write words.Models that generate or process text.glossarymethod:attention-visualisationShowing which words the model looked at, and how hard.Displaying which positions a model attends to, and how strongly.glossarymethod:crosscoderA variant that works across several layers, or several models, at once.A sparse dictionary trained jointly across several layers or several models.glossarymethod:dimensionality-reductionFlattening very high-dimensional data down to two or three dimensions so it can be drawn.Projecting high-dimensional activations into two or three dimensions so their structure can be seen.glossaryphenomenon:induction-headsParts of the model that spot a pattern earlier in the text and continue it.Attention heads that continue a pattern seen earlier in the context.glossaryphenomenon:polysemanticityOne part of the model doing several unrelated jobs at once.A single component responding to several unrelated concepts.glossaryphenomenon:attention-headsThe parts that decide which earlier words matter for the next one.What an attention head does, on a chosen task or across the full data distribution.glossary
- Citation
- Luger, R., Kamath, H., Finkbeiner, D., Goel, P., Jermyn, A., Zimmerman, S., Batson, J. and Conerly, T., 2026. HeadVis: An Interactive Tool For Investigating Attention Heads. Transformer Circuits Thread.
- Related entries
- IA-004
- Confidence
- High
- Source of info
- Read the full article, 2026-08-06
- Date added
- 2026-08-06
- Added by
- Claude
- Notes
- Frontend open-sourced; backends not. Custom-prompt and logit-lens views excluded from the public release. Hosted demos for Haiku 3.5 subset and full Gemma 3 1B. Access classified Open weights on the strength of the public Gemma 3 1B demo; the full Haiku 3.5 version requires internal access.