Tag:lora

I Built an 11,200-Example Synthetic Dataset, Fine-Tuned a Model, and Then Didn't Ship It

A post-mortem on the Dragon Commentary Studio LoRA: what the data pipeline got right, why the fine-tune still lost to the base model, and why shipping the base model was the correct engineering call.

Life After Training: Merging LoRAs, Stacking Them, and Serving a Hundred Fine-Tunes on One GPU

LoRA Deep Dive · Part 4

The first three articles in this series were about making an adapter. This one is about what makes adapters a genuinely different kind of artifact from a fine-tuned model: what you can do with them afterward.

The LoRA Knobs: Rank, Alpha, Targets, and the Settings Everyone Copies Without Understanding

LoRA Deep Dive · Part 3

Every LoRA config file has the same handful of lines — r, alpha, dropout, targetmodules, learning rate — and almost everyone (me included, at first) fills them in by copying a config that worked for someone else.

QLoRA: The Paper That Put Large-Model Fine-Tuning on a Desk

LoRA Deep Dive · Part 2

LoRA solved half the memory problem: with adapters, the trainable state — gradients and optimizer moments — collapses to almost nothing.

LoRA: Why Fine-Tuning a Billion-Parameter Model Only Takes a Few Million Parameters

LoRA Deep Dive · Part 1

Full fine-tuning of a large language model is brutally expensive — not because of the forward pass, but because training multiplies memory.