ChannelPulse
Data science
"Data scientist" is one title over at least four different jobs, and a loop tests whichever one that company means. This track builds the parts that every version shares — the arithmetic of a test you have to size, evaluation that survives a rare positive class, and analysis that isn't quietly reporting an artefact of when the query ran.
Most rejections here are not for getting the statistics wrong. They are for reporting a number without noticing what it could not have told you.
The method — the round you are actually in
Which of the four data-science jobs a loop is hiring for and how to tell before you answer; what a p-value does and does not claim, and how many observations the test you just proposed actually needs; why a model with an excellent ROC-AUC can be useless in production; and the censoring bug that makes flat retention look like a collapse.
-
1
Which interview are you actually in
One title covers at least four jobs, and every company's loop tests a different mix of them. This lesson maps the rounds to the jobs, shows how to identify which mix you are facing before the first question, and covers the take-home — the stage with the highest rejection rate and the least feedback. Free -
2
The two errors, and the size of the test
A p-value answers a question almost nobody asks it, and a sample size follows from four numbers you can state out loud. This lesson pins down what each error rate means, reproduces the NIST handbook's worked sample-size example to prove the arithmetic, and then does the iteration everyone skips. Free -
3
Evaluation that survives a rare positive class
ROC-AUC cannot see class prevalence, which is why a fraud model can score 0.91 and drown its users in false alarms. This lesson holds one model fixed, changes only how many negatives exist, and watches ROC-AUC refuse to move while precision collapses — then turns the threshold into a cost decision. Free -
4
Cohorts that are not lying to you
The most common bug in analytics is not a wrong join — it is asking a 7-day question of cohorts that have existed for three days. This lesson builds a retention table that reports a collapse where nothing happened, fixes it with one predicate, and then pins down the metric definitions that make cohort numbers comparable at all. Free
Experiments — the checks before the result
Almost every wrong experiment call is made before anyone looks at the effect size. This module covers the unit you randomise and how to prove the assignment worked; the four ways a valid test still produces a wrong decision — peeking, novelty, interference, and multiple comparisons; and what to do when a randomised test is impossible and you have to argue causation from observational data.
-
1
Randomisation, sample-ratio mismatch, and guardrails
Before an experiment can be right about its effect it has to be right about its assignment. This lesson picks the unit of randomisation, computes the sample-ratio-mismatch statistic that invalidates a test outright, shows an aggregate check passing while a per-day check fails, and defines the guardrail metrics that stop a winning test from shipping damage. 26 min · app -
2
Peeking, novelty, and interference — four ways a valid test lies
The arithmetic can be right and the decision still wrong. This simulates 2,000 A/A tests to show peeking turning a 5% false-positive rate into 20%, then covers the novelty effect that fades, the interference that puts your treatment inside your control, and the multiple comparisons that manufacture a winner out of nothing. 27 min · app -
3
Causal inference when you cannot run a test
Sometimes randomising is impossible — the feature shipped everywhere, the policy is legally required, the change is a price. This lesson works difference-in-differences by hand, shows the pre-trend check that decides whether the estimate means anything, and lays out matching, synthetic control, instrumental variables, and regression discontinuity with the assumption each one buys. 26 min · app
Modelling and SQL — from a number to a decision
The modelling rounds are less about algorithms than about whether your evaluation is honest and whether your output is usable. This module builds a leaky feature and watches it produce accuracy out of pure noise, converts a probability into a threshold someone can act on, and writes the SQL that answers the question actually asked rather than the one that is easy to query.
-
1
Leakage — how a model gets 99% and helps nobody
Leakage is the failure that looks like success, and it is the most common reason a model that validated well does nothing in production. This builds a dataset with no signal whatsoever and extracts 69% accuracy from it by encoding a feature on the wrong rows, then writes the join that quietly counts events from after the prediction date and the as-of join that fixes it. 26 min · app -
2
Calibration and thresholds — turning a score into a decision
A model outputs a number; a product needs an action. This lesson sweeps a threshold and shows that neither precision nor recall picks it — the cost of the two mistakes does, at exactly COST_FP/(COST_FP+COST_FN) — then builds a reliability table proving that a model with identical ranking can be badly miscalibrated, and that the cost rule silently breaks when it is. 27 min · app -
3
SQL that answers the question you were asked
The SQL round is not testing syntax, it is testing whether your query means what you think it means. This runs three queries that return confident wrong answers — a join that inflates revenue by 84%, a "never redeemed" filter that returns nothing because of one NULL, and a first-purchase query with a tie — and the window-function patterns that get each one right. 28 min · app
Work through it with feedback
Reading the pattern is step one. The app runs you through it — editable code cells, the question bank, and a mock loop that grades your answer.
Open the curriculum in ChannelPulse