AI interviews · AI engineers, the loop and the fundamentals

How large language models work, from tokens to the next token

The most tested topic in AI engineer loops is what happens inside the model you build on. This lesson follows one request through a decoder-only transformer, from tokenisation and position to attention, the block, decoding and the KV cache, then covers context length, scaling laws and mixture of experts, with the numbers you should be able to compute live.

20 min read Full lesson in the app Patterns: prefill-versus-decode, memory-not-flops, compute-optimal-is-not-deploy-optimal

What you'll be able to do

Before this: ml-and-deep-learning-basics

Log in to read the rest of this lesson

The most tested topic in AI engineer loops is what happens inside the model you build on. This lesson follows one request through a decoder-only transformer, from tokenisation and position to attention, the block, decoding and the KV cache, then covers context length, scaling laws and mixture of experts, with the numbers you should be able to compute live. This walkthrough runs about 20 minutes, with runnable code you can edit and re-run as you read. A new account gets 7 days of Pro free, no card required.

Log in to continue

The first module of every track is free to read on the web: see what's open in AI interviews.