Track 13 · Resampling and Statistical Reasoning · Lesson 10 of 10
Reproducibility and ethics
Make an analysis reproducible, report results honestly and in proportion, and protect the privacy of the people behind the data.
A consultant's report says a programme “cut absences by 50%”. Nobody can re-run the analysis: the code is lost, the data were edited by hand, and the random seed was never recorded. The 50% turns out to be a fall from 2 to 1 absence per thousand days, and the published table lets readers identify the one manager in her department. Each of these is a failure of statistical practice, not of arithmetic.
Two words are often confused:
- An analysis is reproducible when someone else, given the same data and code, gets the same numbers.
- A finding replicates when a new study, with new data, reaches the same conclusion.
Reproducibility is the minimum standard: if your own numbers cannot be regenerated, nobody can check them, and errors survive. Replication is how science decides whether a finding is real (see p-hacking and publication bias).
Included with a Formula Quest membership
It has 8 graded exercises with feedback, hints and worked solutions. After it you will be able to:
- Distinguish reproducibility from replication and build an analysis anyone can re-run
- Record seeds and B so simulation-based results can be regenerated exactly
- Report effects with absolute and relative sizes, uncertainty and the limits of the design
- Protect respondents by suppressing small cells and never releasing identifying row-level data