Last time, we said an LLM is just guessing the next word really well. Fair question: how does it guess so well? The answer is an idea called the Transformer, built around a mechanism called attention — and it’s genuinely one of the cleverest tricks in modern computing.
The problem: which words actually matter?
Take this sentence: “The trophy didn’t fit in the suitcase because it was too big.” What does “it” refer to — the trophy, or the suitcase? You solved that instantly. But notice what you actually did: you glanced back across the whole sentence and weighed which earlier word “it” was most connected to.
Older AI models read sentences strictly left to right, one word at a time, like reading through a keyhole — by the time they reached “it,” the word “trophy” had mostly faded from memory. That was the real bottleneck holding language AI back for years.
Attention: look at every word at once
The Transformer’s fix is attention, and the idea is almost embarrassingly simple: let every word glance at every other word, all at the same time, and decide for itself which ones matter most. No reading through a keyhole — the whole sentence is visible at once.
trophy didn’t fit in the suitcase because it was too big
When the model processes “it,” attention lets it link straight back to “trophy” — skipping every irrelevant word in between.
It does this for every single word, simultaneously, building a web of “who’s connected to whom” across the whole sentence. That web is what lets it track pronouns, sarcasm, long paragraphs, even callbacks from several sentences ago.
why this was such a big deal
Reading in a line vs. reading all at once
The old way (RNNs)
Words processed one by one, in strict order. Earlier context quietly fades the further along it gets.
Slow, and forgetful over long sentences.
Attention (Transformers)
Every word connects to every other word at once. Nothing fades — distance stops being a disadvantage.
Faster to train, and dramatically better at long context.
This one shift — published in a 2017 paper literally titled “Attention Is All You Need” — is the actual foundation underneath ChatGPT, Claude, Gemini, and basically every major AI model since. The “T” in GPT stands for Transformer.
not just for words
It’s not just for sentences anymore
“Pay attention to every relevant piece at once” turned out to be useful for more than words. The same core idea — just aimed at pixels instead of tokens — is part of what powers today’s image generators, and it shows up in music, video, and even protein-folding models. One idea, reused everywhere.
where you've met it already
- Chatbots — Tracking who “he,” “she,” or “it” means across a long conversation
- Translation — Matching grammar rules that jump around between languages
- Summarizing — Spotting which sentence in paragraph one still matters by paragraph ten
- Image generators — Linking “a red umbrella” in your prompt to the right pixels