IA-011 / 2024
TalkTuner is a dashboardA single screen putting several different readouts side by side.Several visual forms arranged together as one interface, read as a unit.glossary showing a user modelThe picture a model builds of who it is talking to, such as age, gender or mood.What the model has inferred about the person using it.glossary, about biasThe model treating people or groups differently in ways it should not.Systematic differential treatment of people or groups by the model.glossary and user-modellingWhat the model has quietly worked out about the person talking to it.The model's internal inferences about the person it is talking to.glossary.
- ID
- IA-011
- Name
- TalkTuner — dashboard for transparency and control of conversational AI
- Artefact type
- Dashboard
- Visual form
- DashboardA single screen putting several different readouts side by side.Several visual forms arranged together as one interface, read as a unit.glossary
- Makes visible
- User modelThe picture a model builds of who it is talking to, such as age, gender or mood.What the model has inferred about the person using it.glossary
- Role of the visual
- Instrument
- Runnable by a visitor
- Unknown
- Date published
- 2024-06-12
- Year
- 2024
- Link status
- Live
- Built by (people)
- Yida Chen; Aoyu Wu; Trevor DePodesta; Catherine Yeh; Kenneth Li; Nicholas Castillo Marin; Oam Patel; Jan Riecke; Shivam Raval; Olivia Seow; Martin Wattenberg; Fernanda Viégas
- Organisation
- Harvard University
- Venue / published in
- arXiv (v3, October 2024)
- Sector
- Academia
- Country / region
- US
- Open source
- Unknown
- Method / technique
- Linear probes over internal activations to extract a 'user model', displayed live and made editable so the user can intervene on it
- Model(s) studied
- An open-source LLM (specific model not named in the abstract)
- Access required
- Open weights
- What it visualises
- What the chatbot has internally inferred about you — age, gender, education level, socioeconomic status — shown in real time beside the conversation, with controls to change it
- Interaction affordances
- Converse normally while watching the inferred user model update; edit those inferences directly and watch the system's behaviour change
- Pragmatic vs Basic science
- Pragmatic
- Reverse-eng vs Concept-based
- Concept-based
- Observational vs Interventional
- Both
- Intended audience
- General public
- Description (card)
- A dashboard that sits beside a chatbot and shows, in real time, what the model has internally inferred about the user's age, gender, education and socioeconomic status — and lets the user edit those inferences and watch the responses change.
- Why it matters
- Almost everything else in this ledger is built for researchers; this is built for the person being modelled, and a user study found it helped participants expose the system's biased behaviour.
- Visual / design notes
- Dashboard-beside-chat is the key move, but making the user model editable rather than merely visible is what turns transparency into control.
- Tags
- modality:languageText. Models that read and write words.Models that generate or process text.glossarymethod:probingTraining a small, simple classifier to test whether some information is present inside the model.Training a simple classifier on internal activations to test what they encode.glossarymethod:steeringNudging the model's internals mid-thought to change what it says.Adding or subtracting a direction in activation space in order to alter behaviour.glossaryphenomenon:biasThe model treating people or groups differently in ways it should not.Systematic differential treatment of people or groups by the model.glossaryphenomenon:user-modellingWhat the model has quietly worked out about the person talking to it.The model's internal inferences about the person it is talking to.glossary
- Citation
- Chen, Y., Wu, A., DePodesta, T., Yeh, C., Li, K., Castillo Marin, N., Patel, O., Riecke, J., Raval, S., Seow, O., Wattenberg, M. and Viégas, F., 2024. Designing a Dashboard for Transparency and Control of Conversational AI. arXiv:2406.07882
- Related entries
- IA-010
- Confidence
- High
- Source of info
- Read the arXiv abstract page in full, 2026-08-06. Project page (bit.ly/talktuner-project-page) not fetched.
- Date added
- 2026-08-06
- Added by
- Claude
- Notes
- Cited in Sharkey et al. 2025 section 3.7 on human-computer interaction with model internals. phenomenon:bias and phenomenon:user-modelling were proposed for this row and promoted into the controlled vocabulary by Ava on 2026-08-06; both are now in use here.