IA-009 / 2024
Transformer Explainer is a flow diagramBoxes joined by arrows, showing what happens in what order.Boxes and arrows showing a process or structure in sequence.glossary showing architectureThe shape of the model itself: what parts it has, and the order they run in.The model's structure rather than anything it learned.glossary, about attention-headsThe parts that decide which earlier words matter for the next one.What an attention head does, on a chosen task or across the full data distribution.glossary.
- ID
- IA-009
- Name
- Transformer Explainer
- Artefact type
- Interactive tool
- Visual form
- Flow diagramBoxes joined by arrows, showing what happens in what order.Boxes and arrows showing a process or structure in sequence.glossary
- Makes visible
- ArchitectureThe shape of the model itself: what parts it has, and the order they run in.The model's structure rather than anything it learned.glossary
- Role of the visual
- Explanation
- Runnable by a visitor
- On your own inputs
- Date published
- 2024-08-08
- Year
- 2024
- Link status
- Live
- Built by (people)
- Aeree Cho; Grace C. Kim; Alexander Karpekov; Alec Helbling; Zijie J. Wang; Seongmin Lee; Benjamin Hoover; Duen Horng (Polo) Chau
- Organisation
- Georgia Institute of Technology (Polo Club of Data Science)
- Venue / published in
- IEEE VIS 2024; arXiv
- Sector
- Academia
- Country / region
- US
- Open source
- Yes
- Method / technique
- Live in-browser execution of a model with every stage of the forward pass visualised, and smooth transitions between abstraction levels
- Model(s) studied
- GPT-2
- Access required
- Open weights
- What it visualises
- The full GPT-2 forward pass — embeddings, attention queries keys and values, MLP, and next-token probabilities — updating live as you type
- Interaction affordances
- Type your own input and watch predictions update in real time; expand any stage down to the underlying matrix operations; adjust temperature; no installation or GPU required
- Pragmatic vs Basic science
- Neither / educational
- Reverse-eng vs Concept-based
- Neither
- Observational vs Interventional
- Observational
- Intended audience
- General public; Students
- Description (card)
- An interactive tool that runs a live GPT-2 instance in your browser and visualises every stage of the forward pass — embeddings, attention, MLP, output probabilities — updating as you type your own text.
- Why it matters
- The most accessible route into transformer internals that exists: no installation, no GPU, no prior knowledge. That is a different kind of contribution from any research result in this ledger, and arguably a wider-reaching one.
- Visual / design notes
- The core design idea is smooth movement between abstraction levels — you can zoom from a block diagram down to the matrix multiplication without losing your place.
- Tags
- modality:languageText. Models that read and write words.Models that generate or process text.glossarymethod:attention-visualisationShowing which words the model looked at, and how hard.Displaying which positions a model attends to, and how strongly.glossaryphenomenon:attention-headsThe parts that decide which earlier words matter for the next one.What an attention head does, on a chosen task or across the full data distribution.glossary
- Citation
- Cho, A., Kim, G.C., Karpekov, A., Helbling, A., Wang, Z.J., Lee, S., Hoover, B. and Chau, D.H., 2024. Transformer Explainer: Interactive Learning of Text-Generative Models. IEEE VIS 2024. arXiv:2408.04619
- Related entries
- IA-010
- Confidence
- High
- Source of info
- Read the arXiv abstract page in full, 2026-08-06. The live tool itself was not inspected — it is client-rendered and the fetch returned an empty page shell.
- Date added
- 2026-08-06
- Added by
- Claude
- Notes
- Educational rather than a research discovery: the architecture it visualises is known, not uncovered. A boundary case for the inclusion rule, logged because the visualisation IS the contribution and it was peer-reviewed at IEEE VIS. Access classified Open weights: GPT-2 runs locally in the visitor's browser, no server or account needed.