Start a project

Fig. I — A run, in order

Building Blocks

14 parts, read in order — each one assumes the one before it, and 5are still being written. Back toall runs orthe full archive.

Start at the beginning

The parts

Somewhere between "it's just predicting the next word" and a hundred-billion-dollar training run sits an actual mechanism. This series builds it, piece by piece, starting from the smallest one there is.

  1. Twelve Pieces, One Working Model
  2. The billions of knobs inside a model
  3. Turning meaning into a list of numbers
  4. Embeddings: where similar ideas learn to live near each other
  5. Dimensions: why AI thinks in thousands of directions at once
  6. Tensors: the shape data takes inside AI
  7. Tokens, Revisited: how text becomes math
  8. Neural Network Layers: Stacking Weighted Sums With a Foldin progress
  9. Inside the Transformer: How Tokens Read Context From Each Otherin progress
  10. The Same Model, Two Very Different Jobs
  11. Loss Functions & Gradient Descent: How a Model Finds Out It Was Wrongin progress
  12. Small Models, Big Models & vLLM: Sizing One and Serving Itin progress
  13. Real World AI: Every Mechanism, Checked Against What's Shippingin progress
  14. You Built the Whole Chain