Post-Training 101

Post-training a small open model to write SQL, one technique at a time, in public. The code is on GitHub.

  1. Understanding the data and the benchmark None yet: the data first
  2. The starting line: a 7B against the frontier How answers are scored; 95% intervals
  3. What is each prompt ingredient worth? planned Testing each change for significance; repetition penalty on logits
  4. How good are frontier models on the same prompt? planned Tokens to dollars; cost per correct answer
  5. Supervised fine-tuning planned Cross-entropy loss; LoRA’s low-rank update
  6. Distilling reasoning from a larger model planned Rejection sampling; distillation as minimising KL divergence
  7. More samples, better answers: voting and pass@k planned Majority-vote probability; pass@k
  8. Direct Preference Optimization planned The Bradley-Terry model; deriving the DPO loss
  9. Reinforcement learning with GRPO planned Policy gradient; the group-relative advantage
  10. The final exam: BIRD’s hidden test set planned What carries over from dev to test