Fig. I — Filed by category

Category:Reinforcement Learning

4 meditations filed under Reinforcement Learning. Back to all categories orthe full archive.

Clippings

RLHF and Beyond: How Reinforcement Learning Taught Language Models to Behave

Reinforcement Learning · Part 4

A base language model fresh out of pretraining is a strange creature. It has read a large fraction of the internet and can continue any text with uncanny fluency — but it isn't trying to help you.

Read entry

Vol. IV · 24

Reinforcement Learning

Policy Gradients and PPO: Learning Behavior Directly

Reinforcement Learning · Part 3

DQN taught me one way to act intelligently: learn the value of every action, then pick the best one. But there's a second lineage in reinforcement learning with the opposite philosophy — skip the values, and optimize the behavior itself.

Read entry

Vol. IV · 23

Reinforcement Learning

DQN: The Moment Reinforcement Learning Met Deep Learning

Reinforcement Learning · Part 2

In 2013, a small London startup called DeepMind posted a paper showing a single algorithm learning to play Atari games — from raw pixels, with no game-specific knowledge, using only the score as feedback.

Read entry

Vol. IV · 22

Reinforcement Learning

Reinforcement Learning from Zero: Agents, Rewards, and the Loop That Learns from Consequences

Reinforcement Learning · Part 1

Supervised learning felt intuitive to me from day one: here's the input, here's the right answer, minimize the difference.

Read entry