AI interviews · Building with LLMs, retrieval, tuning, agents and other inputs

Fine-tuning and alignment, from LoRA to preference tuning

When training a model beats prompting or retrieval, how supervised fine-tuning and LoRA work down to the parameter count, and how RLHF, DPO and GRPO shape behaviour. It covers the data, memory and evaluation questions interviewers use to separate people who have run a fine-tune from people who have read about one.

18 min read Full lesson in the app Patterns: behaviour-not-knowledge, lora-arithmetic, kl-leash, differential-eval

What you'll be able to do

Before this: how-llms-work, rag-and-retrieval

Log in to read the rest of this lesson

When training a model beats prompting or retrieval, how supervised fine-tuning and LoRA work down to the parameter count, and how RLHF, DPO and GRPO shape behaviour. It covers the data, memory and evaluation questions interviewers use to separate people who have run a fine-tune from people who have read about one. This walkthrough runs about 18 minutes, with runnable code you can edit and re-run as you read. A new account gets 7 days of Pro free, no card required.

Log in to continue

The first module of every track is free to read on the web: see what's open in AI interviews.