LLM Training Stages

Planned topics from the course outline:

• Next-token pretraining, supervised fine-tuning, and instruction tuning.
• Reward modeling, RLHF, RLAIF, direct preference optimization, and reinforcement learning for reasoning.

Lecture materials

• Lecture 05 — Final Project Directions — September 28, 2026. The project handout reconstructed from screenshots, with an English text, an English overview, and reference links.

Further reading

• AI Safety (CS 2881), Fall 2026 — includes lectures on modern LLM training and RL post-training.
• Appendix C: Reinforcement Learning — introductory RL concepts relevant to post-training.
• ILIAD Intensive Curriculum — broader alignment curriculum for placing training choices in context.