Tag:distillation

Optimizing a Model: The Complete Guide to Distillation, Quantization, Pruning, and Beyond

Training a capable model is only half the battle. The model that comes out of pretraining or fine-tuning is almost never the model you actually want to ship.

Synthetic Data: Training Models on Text That Models Wrote

Data Engineering · Part 3

There's an idea that would have sounded like a joke a few years ago: take a language model, have it write its own training data, and train the next model on it.