Vol. IV · 36
Why "just the data" isn't neutral
Foundations · Part 7
AI doesn't invent its opinions from nowhere. It learns them — including the ones we'd rather it didn't — from us.
Read entryFig. I — Filtered by tag
Tag:training data
5 meditations tagged training data. Back toall tags or the full archive.
Clippings
Vol. IV · 13
Building a Fine-Tuning Dataset: The Practical Craft Nobody Writes Papers About
Data Engineering · Part 4
The first three articles in this series were about the grand stuff — trillion-token corpora, scaling economics, synthetic generation.
Read entryVol. IV · 12
Synthetic Data: Training Models on Text That Models Wrote
Data Engineering · Part 3
There's an idea that would have sounded like a joke a few years ago: take a language model, have it write its own training data, and train the next model on it.
Read entryVol. IV · 11
Quality vs. Quantity: The Data Question That Keeps Changing Its Answer
Data Engineering · Part 2
Every training run begins with a budget question: you have finite compute and finite data — what maximizes capability?
Read entryVol. IV · 10
From Raw Web to Training Corpus: The Unglamorous Pipeline Behind Every LLM
Data Engineering · Part 1
When I started learning ML, I assumed the hard part was the model — architectures, optimizers, all the math I've written about so far.
Read entry