Track 8 · Hypothesis Testing · Lesson 11 of 14
Effect sizes
Quantify how large an effect is — in original units and with standardised measures such as Cohen's d, risk difference and relative risk — and judge practical significance with a confidence interval.
A trial reports p < 0.0001. Is the drug a breakthrough or a marginal improvement? The p-value cannot say: it mixes the size of the effect with the size of the study. Decision-makers need to know how big the effect is and how precisely it is known.
An effect size describes the magnitude of an effect, independent of sample size. The most useful is usually the raw (unstandardised) effect in the original units — “9.5 mmHg lower”, “2.7 minutes faster”, “0.26 percentage points more conversions” — because people can judge it against what matters in practice. Standardised effect sizes put different measures on a common scale, which helps when units are arbitrary (test scores, ratings) and for power analysis.
Included with a Formula Quest membership
It has 7 graded exercises with feedback, hints and worked solutions. After it you will be able to:
- Report effects in original units with a confidence interval first
- Compute Cohen's d, d_z and Hedges' g, and risk difference, relative risk and odds ratio
- Judge practical significance against a minimum important difference using the interval
- Explain why a small p-value does not mean a large effect