Skip to content

attention

attention

Plot ordinary transformer attention, including expanded fork streams.

Reuse the standard heatmap analysis on the checkpoint's cached batch. Select only transformer attention (not Scratchbubbles' selector), and expand the padding mask by token_index before the compiled masked-softmax calculation. The inherited plot method renders the gathered CPU array.