Tag:fine-tuning

I Built an 11,200-Example Synthetic Dataset, Fine-Tuned a Model, and Then Didn't Ship It

A post-mortem on the Dragon Commentary Studio LoRA: what the data pipeline got right, why the fine-tune still lost to the base model, and why shipping the base model was the correct engineering call.

The LoRA Knobs: Rank, Alpha, Targets, and the Settings Everyone Copies Without Understanding

LoRA Deep Dive · Part 3

Every LoRA config file has the same handful of lines — r, alpha, dropout, targetmodules, learning rate — and almost everyone (me included, at first) fills them in by copying a config that worked for someone else.

QLoRA: The Paper That Put Large-Model Fine-Tuning on a Desk

LoRA Deep Dive · Part 2

LoRA solved half the memory problem: with adapters, the trainable state — gradients and optimizer moments — collapses to almost nothing.

LoRA: Why Fine-Tuning a Billion-Parameter Model Only Takes a Few Million Parameters

LoRA Deep Dive · Part 1

Full fine-tuning of a large language model is brutally expensive — not because of the forward pass, but because training multiplies memory.

Building a Fine-Tuning Dataset: The Practical Craft Nobody Writes Papers About

Data Engineering · Part 4

The first three articles in this series were about the grand stuff — trillion-token corpora, scaling economics, synthetic generation.