LLM Attention Visualization
AI
model visualization
SHOW HN2026
How HN took it
9 positive, 5 neutral or asking, 0 mixed, 3 critical, out of 17 comments about the project. Thread score +0.30.
Idea: 6 for, 1 againstDesign: 1 for, 2 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Clear, intuitive way to understand attention
From 3 comments
This is great, thank you. I have to teach this stuff on Friday so perfect timing. It's hard to explain the attention mechanism in a way that becomes intuitive because the…
MCP123 on HNThis is great, I've read multiple books and watched videos about the attention mechanism. Now that I understand it, this is the clearest example I've seen on how attention works.
fuddle on HNThis is very cool. It's simpler than Bertviz for understanding inference and surface level and a good starting people for new learners as well.
sshussain270 on HN
Simple and visualizable
From 2 comments
thanks for making it simple and visualizable
talhaanwar on HNI like the visualisation. Pretty cool
itsnasme on HN
Objections
Weighting by value vector magnitude is a simplistic measure of influence
From 2 comments
I highly question this simplistic idea of high vector magnitude = high influence.
sva_ on HNI don't disagree with that. I did add an entire caveat paragraph there. To me, it's more of a neat visualization, not something that can be used to interpret LLM behavior. Even…
ifz on HN
Requests
Pause should pause playback instead of resetting
From 2 comments
UX report. I wished to examine attention state step by step, but I found the animation moved along too fast for that. So I tried pausing... On Chromium/linux, pressing pause…
mncharity on HNThe pause-on-load problem is a real UX trap. Stepping through one token at a time is how people actually learn attention; single-step controls would help more than autoplay.
Mugshelf on HN
Single-step controls
From 2 comments
UX report. I wished to examine attention state step by step, but I found the animation moved along too fast for that. So I tried pausing... On Chromium/linux, pressing pause…
mncharity on HNThe pause-on-load problem is a real UX trap. Stepping through one token at a time is how people actually learn attention; single-step controls would help more than autoplay.
Mugshelf on HN
Controls to limit which layers are summed
From 1 comment
Hmm, I might try to add some controls to limit which layers get summed up. It might be able to reveal more patterns. Right now only simple correlations are visible.
ifz on HN
Per-head color layers instead of summed view
From 1 comment
Might be cool to try different colors for the different attention heads instead of summing them across all attention heads, and making the backgrounds composed of stacked color…
nullbio on HN
Questions
What is the actual formula
From 2 comments
I am curious what's the actual formula. I mean, there so many headers and layers, it is tricky to make a choice that will resonate with our intuition . Is it some weighted…
stared on HNHow it works?
asd000hh on HN
Whether later layers get drowned out by earlier ones
From 1 comment
neat, combining info from two phrases is hard to see without such a tool. are you worried later-layer attention gets drowned out by earlier layers just because there are more of…
wopak on HN
Alternatives named
BertViz
From 1 comment
This is very cool. It's simpler than Bertviz for understanding inference and surface level and a good starting people for new learners as well.
sshussain270 on HN
Transformer circuits attention QK analysis
From 1 comment
You get what you pay for. If you want to think harder and get more, https://transformer-circuits.pub/2025/attention-qk/index.html
smallmancontrov on HN