How large language models work, from tokens to the next token
The most tested topic in AI engineer loops is what happens inside the model you build on. This lesson follows one request through a decoder-only transformer, from tokenisation and position to attention, the block, decoding and the KV cache, then covers context length, scaling laws and mixture of experts, with the numbers you should be able to compute live.
What you'll be able to do
- Explain tokenisation, positional encoding and scaled dot-product attention well enough to sketch them on a whiteboard
- Describe a pre-norm decoder block and say where its parameters and compute go
- Compare greedy, temperature, top-k, top-p and min-p decoding and pick settings for a given task
- Compute the KV cache size for a model and explain why GQA, paged caches and speculative decoding exist
- Defend a position on context length, Chinchilla scaling and mixture of experts with the trade-offs stated
Before this: ml-and-deep-learning-basics
Log in to read the rest of this lesson
The most tested topic in AI engineer loops is what happens inside the model you build on. This lesson follows one request through a decoder-only transformer, from tokenisation and position to attention, the block, decoding and the KV cache, then covers context length, scaling laws and mixture of experts, with the numbers you should be able to compute live. This walkthrough runs about 20 minutes, with runnable code you can edit and re-run as you read. A new account gets 7 days of Pro free, no card required.
Log in to continueThe first module of every track is free to read on the web: see what's open in AI interviews.