Track 14 · Practical Data Analysis · Lesson 11 of 11
Reproducible analysis
Record an analysis so that anyone — including you in six months — can re-run it from the raw file and get the same numbers.
Three months after a report, finance asks why your average customer spend (5 481 SAR) differs from theirs (12 128 SAR). If you kept the raw file, a numbered list of steps with row counts, and the settings you used, the answer takes five minutes. If not, nobody can tell which number is right.
An analysis is reproducible when someone else, starting from the same raw data and following your written steps, gets exactly the same results. It is the minimum standard for a number that will be used in a decision: if a result cannot be re-created, it cannot be checked, corrected or updated when new data arrive.
Reproducibility does not require code. It requires discipline about four things: the source, the steps, the settings and the outputs.
Included with a Formula Quest membership
It has 7 graded exercises with feedback, hints and worked solutions. After it you will be able to:
- Keep the raw file untouched and record its source, version and row count
- Log each cleaning rule as a checkable condition, an action, a count and a reason
- Record filters, method settings (quantile type, confidence level) and random seeds
- Reconcile two different results by comparing inputs and steps, not by arguing about the outputs