Vol. IV · 31
The idea that changed everything
Foundations · Part 2
One clever trick called "attention" is the real reason AI suddenly got good. Here's the whole idea, no math required.
Read entryFig. I — Filtered by tag
Tag:transformers
3 meditations tagged transformers. Back toall tags or the full archive.
Clippings
Vol. IV · 30
What's actually inside ChatGPT?
Foundations · Part 1
Not a brain. Not magic.
Read entryVol. IV · 7
The Transformer, Explained the Way I Wish Someone Had Explained It to Me
ML Foundations · Part 2
In 2017, a paper with the almost arrogant title "Attention Is All You Need" replaced decades of accumulated architecture research with one mechanism.
Read entry