The Gradient Is a Covariance

Published 19 October 2026

Reinforcement learning, on-policy distillation, and near-policy supervised fine-tuning can alter only a small fraction of parameters. The expected gradient is a covariance, while the Fisher is a variance that bounds gradients for any reward. Centering scores under the sampling distribution, not on-policy sampling itself, is the key requirement.