Post-Training 101
Post-training a small open model to write SQL, one technique at a time, in public. The code is on GitHub.
- Understanding the data and the benchmark None yet: the data first
- The starting line: a 7B against the frontier How answers are scored; 95% intervals
- What is each prompt ingredient worth? planned Testing each change for significance; repetition penalty on logits
- How good are frontier models on the same prompt? planned Tokens to dollars; cost per correct answer
- Supervised fine-tuning planned Cross-entropy loss; LoRA’s low-rank update
- Distilling reasoning from a larger model planned Rejection sampling; distillation as minimising KL divergence
- More samples, better answers: voting and pass@k planned Majority-vote probability; pass@k
- Direct Preference Optimization planned The Bradley-Terry model; deriving the DPO loss
- Reinforcement learning with GRPO planned Policy gradient; the group-relative advantage
- The final exam: BIRD’s hidden test set planned What carries over from dev to test