Attention
The idea under every model that writes or draws, reduced to bars.
In 2017 Vaswani and co-authors published Attention Is All You Need. That paper is the core of current AI.
Vaswani and co-authors, 2017, "Attention Is All You Need"
License on this page
License notice
The bars are an original sketch of scaled dot-product attention, not a figure from the 2017 paper. A lawyer cleared this listing.
In 2017 Vaswani and co-authors published Attention Is All You Need, and the idea in it now sits under nearly every model that writes, draws, or answers. It threw out the recurrent step, letting each position look at all the others at once.
The bars are an original sketch of scaled dot-product attention, not a figure from the paper. A query is compared with the keys, the scores are scaled and softened into weights, and the weights mix the values. Several heads run that blend in parallel.
Next piece: Apollo 11 ignition