Statistics Quest

Hypothesis Testing

Test claims with data and draw defensible conclusions.

Hypotheses, test statistics, p-values, significance, Type I and II errors, power, effect sizes, assumptions and multiple testing; one-sample, two-sample, paired and proportion tests — in a workbench where you choose the test, check assumptions, run it and communicate the result.

Lessons

  1. 1The logic of a hypothesis testExplain how a hypothesis test weighs data as evidence against a sceptical claim, and what its two possible conclusions do and do not mean.25 minFree
  2. 2Null and alternative hypothesesWrite null and alternative hypotheses about a population parameter, and choose a one- or two-sided alternative before seeing the data.25 minMembers
  3. 3Test statistics and p-valuesCompute a test statistic as a standardised distance from H0, find its p-value for a one- or two-sided alternative, and interpret the p-value correctly.30 minMembers
  4. 4Significance level and errorsUse a significance level to make a decision, and explain the two kinds of error a test can make and how α, β and power trade off.30 minMembers
  5. 5One-sample t-testCarry out a one-sample t-test from start to finish — hypotheses, assumptions, t, df, p-value, confidence interval and effect size — and state the conclusion in context.35 minMembers
  6. 6Tests for a proportionTest a claim about a single population proportion with the one-sample z-test or the exact binomial test, and report the result with a confidence interval.30 minMembers
  7. 7Two-sample t-testCompare the means of two independent groups with Welch's t-test, report the difference with a confidence interval, and know when the pooled version applies.35 minMembers
  8. 8Paired t-testRecognise paired data, analyse it with a paired t-test on the differences, and explain why pairing can make a test far more powerful.35 minMembers
  9. 9Comparing two proportionsCompare two independent proportions with the two-proportion z-test, estimate the difference with a confidence interval, and interpret an A/B test result in business terms.35 minMembers
  10. 10Power and sample sizeExplain what determines the power of a test, compute power for simple cases, and choose a sample size that gives adequate power before collecting data.35 minMembers
  11. 11Effect sizesQuantify how large an effect is — in original units and with standardised measures such as Cohen's d, risk difference and relative risk — and judge practical significance with a confidence interval.30 minMembers
  12. 12Checking assumptionsCheck the independence, normality and equal-variance conditions of t-tests with plots and judgement, understand how robust the tests are, and choose a sensible alternative when a condition fails.35 minMembers
  13. 13Multiple testingExplain why running many tests inflates false positives, and control the family-wise error rate (Bonferroni, Holm) or the false discovery rate (Benjamini–Hochberg).30 minMembers
  14. 14Reporting a test responsiblyWrite a complete, accurate report of a hypothesis test — question, method, estimate with confidence interval, test result, effect size, assumptions and a conclusion in context — without overclaiming.30 minMembers