Elasticsearch and the search index beside your database
A search engine answers "which documents best match these words" by precomputing an inverted index, which makes it superb at relevance and filtering and a poor place to keep the only copy of your data, so the design question is always how the index stays in step with the database that owns the truth.
What you'll be able to do
- Explain an inverted index, an analyzer, and how BM25 ranks matches, well enough to predict a missed or misranked result
- Keep a search index in sync with the system of record using CDC, and handle out-of-order updates
- Size primary shards and replicas, and explain why the primary count is hard to change
- Paginate deep result sets and name the common ways a search cluster falls over
Before this: picking-a-datastore, kafka-and-event-streams
Log in to read the rest of this lesson
A search engine answers "which documents best match these words" by precomputing an inverted index, which makes it superb at relevance and filtering and a poor place to keep the only copy of your data, so the design question is always how the index stays in step with the database that owns the truth. This walkthrough runs about 22 minutes, with runnable code you can edit and re-run as you read. A new account gets 7 days of Pro free, no card required.
Log in to continueThe first module of every track is free to read on the web: see what's open in System design.