Vol. IV · 25
RLHF and Beyond: How Reinforcement Learning Taught Language Models to Behave
Reinforcement Learning · Part 4
A base language model fresh out of pretraining is a strange creature. It has read a large fraction of the internet and can continue any text with uncanny fluency — but it isn't trying to help you.
Read entry