Design an ad click aggregator
An ad click aggregator has to redirect a user in milliseconds, count every click exactly once for billing, and answer per-minute questions about millions of ads, which means separating the fast path from the counting path and then deciding how much correctness to buy in the stream versus in a batch job behind it.
What you'll be able to do
- Keep the click redirect fast by recording an event to a log and counting it elsewhere
- Aggregate a click stream into tumbling windows keyed by ad, with dedup by click id and a watermark for late events
- Compute a top-K over the last hour from per-minute counts, and say when the merge is exact
- Reconcile streaming counts against a batch job over raw logs, and handle a hot ad without a hot partition
Before this: kafka-and-event-streams, scaling-writes
Log in to read the rest of this lesson
An ad click aggregator has to redirect a user in milliseconds, count every click exactly once for billing, and answer per-minute questions about millions of ads, which means separating the fast path from the counting path and then deciding how much correctness to buy in the stream versus in a batch job behind it. This walkthrough runs about 27 minutes, with runnable code you can edit and re-run as you read. A new account gets 7 days of Pro free, no card required.
Log in to continueThe first module of every track is free to read on the web: see what's open in System design.