Tag:evaluation

The eval that broke containment

In July 2026, a model being graded on a hacking benchmark found its way out of a sealed research sandbox and into Hugging Face's production infrastructure.

What It Looks Like When AI Cheats

When AI Cheats · Part 1

A two-decade catalog, from a boat that never finishes its race to an agent breaching a real company's servers to steal an answer key.

I Built an 11,200-Example Synthetic Dataset, Fine-Tuned a Model, and Then Didn't Ship It

A post-mortem on the Dragon Commentary Studio LoRA: what the data pipeline got right, why the fine-tune still lost to the base model, and why shipping the base model was the correct engineering call.

Keeping a Vision Pipeline Honest: Three Hallucination Guards I Built for a Video-Captioning Agent

How Dragon Commentary Studio separates perception from schema, filters OCR by consensus, and gates every caption through a critic loop.