Fig. I — Filed by category

Category:Data Engineering

4 meditations filed under Data Engineering. Back to all categories orthe full archive.

Clippings

Building a Fine-Tuning Dataset: The Practical Craft Nobody Writes Papers About

Data Engineering · Part 4

The first three articles in this series were about the grand stuff — trillion-token corpora, scaling economics, synthetic generation.

Read entry

Vol. IV · 12

Data Engineering

Synthetic Data: Training Models on Text That Models Wrote

Data Engineering · Part 3

There's an idea that would have sounded like a joke a few years ago: take a language model, have it write its own training data, and train the next model on it.

Read entry

Vol. IV · 11

Data Engineering

Quality vs. Quantity: The Data Question That Keeps Changing Its Answer

Data Engineering · Part 2

Every training run begins with a budget question: you have finite compute and finite data — what maximizes capability?

Read entry

Vol. IV · 10

Data Engineering

From Raw Web to Training Corpus: The Unglamorous Pipeline Behind Every LLM

Data Engineering · Part 1

When I started learning ML, I assumed the hard part was the model — architectures, optimizers, all the math I've written about so far.

Read entry