Fine-tuning and alignment, from LoRA to preference tuning
When training a model beats prompting or retrieval, how supervised fine-tuning and LoRA work down to the parameter count, and how RLHF, DPO and GRPO shape behaviour. It covers the data, memory and evaluation questions interviewers use to separate people who have run a fine-tune from people who have read about one.
What you'll be able to do
- Decide between prompting, retrieval and fine-tuning for a measured gap, and defend the choice
- Explain supervised fine-tuning, loss masking and chat templates, and the silent bugs each one causes
- Compute the parameter count and training memory for LoRA, QLoRA and full fine-tuning
- Compare RLHF, DPO and GRPO, and name the reward hacking each one invites
- Design a before-and-after evaluation that catches forgetting, over-refusal and contamination
Before this: how-llms-work, rag-and-retrieval
Log in to read the rest of this lesson
When training a model beats prompting or retrieval, how supervised fine-tuning and LoRA work down to the parameter count, and how RLHF, DPO and GRPO shape behaviour. It covers the data, memory and evaluation questions interviewers use to separate people who have run a fine-tune from people who have read about one. This walkthrough runs about 18 minutes, with runnable code you can edit and re-run as you read. A new account gets 7 days of Pro free, no card required.
Log in to continueThe first module of every track is free to read on the web: see what's open in AI interviews.