Attention Spectrometer
The Attention Spectrometer probes how a query attends to a document. Select a point, type a question, and Mantis captures one attention tensor from the model, then derives every view in the browser from that single capture: a layer-by-head grid, a thermal map of the document text, a ranked list of heavy hitters, and a per-token spectrum showing which heads and layers light that token up.
Where to find it
Open View > Inspection > Attention Spectrometer.
Unlike the rest of the Inspection panels, this one is map-agnostic: it is not wrapped in a map instance and it does not wait on map metadata. It still reads your selection to find its document.
What it needs
The document is the selected point’s text, taken strictly from the space’s semantic (text) columns and joined together. Nothing else from the metadata is sent.
- With no point selected, the document box says “Select a point in the space to load its document.”
- With a point that has no semantic fields, it says “(no semantic / text fields on this point)”.
- With several points selected, only the first is probed, and the panel says so: single-document mode.
Selecting a point, or its document finishing loading, probes it automatically with the current query. Editing the query does not re-probe; press Probe attention or Ctrl+↵.
Controls
Query is the probe, a free-text question, defaulting to “What is this document about?”.
Head grid is one cell per attention head, laid out layers down and heads across, colored by how much filtered mass that head sends to the document. Hover a cell to preview that head alone; click to pin it as the only active head; Cmd, Ctrl or Shift click to add or remove a head from the summed aggregate. Three buttons above it select all, none, or the heaviest top 10% of heads. The line underneath reports either the previewed head or how many of the total heads are currently summed.
Filters & scale shows two filters that are always on: the attention sink (position 0 and pure punctuation) is excluded, and only content words count, using a vendored NLTK English stopword list. log scale switches the color mapping to logarithmic, which is what you want when one token dominates. chunk size is a slider from single tokens up to the whole document; above zero, tokens are grouped into spans of that many tokens and the thermal map colors spans instead of tokens.
Scale is the heat ramp legend, labelled with the lowest and highest unit mass as a percentage of the selected total.
Thermal map is the document itself, each token or span colored by the attention it receives from the currently active heads. Filtered-out tokens stay grey. Click any unit to read its spectrum; click it again to deselect.
Heavy hitters ranks the top twelve units by mass, with a bar and a percentage. Clicking a row selects that unit.
Spectrum appears once a unit is selected, and shows its share of the selected mass, its top six heads as L<layer>·H<head> bars, and a per-layer histogram from L0 to the last layer.
The header shows the model name, the tensor shape as layers x heads and token count, and, when the model returns them, the attention entropy and top-k coverage.
- Select one point on the map.
- Read the document box to confirm the text Mantis picked up.
- Type a question in the query box and press Probe attention.
- Narrow the heads. Start with top 10%, then click individual cells in the grid to isolate one head at a time.
- Click a hot token in the thermal map to see which heads and layers are responsible for it.
Limits
- One document per probe. Multi-point selections use the first point only.
- If the model host is unreachable, the panel does not fail; it builds a deterministic synthetic tensor of the same shape so the views stay interactive, marks the header synthetic, and shows the underlying error. Do not read conclusions from a synthetic capture.
- The sink and content-word filters cannot be turned off.
- The capture happens once per document and query. Changing heads, chunk size or scale is pure browser math over the tensor you already have.