Resampling and Statistical Reasoning skill check
A timed check of bootstrap and permutation inference, Monte Carlo error, robust statistics, missing data, sensitivity analysis, Simpson's paradox, selection bias, multiple testing and honest, reproducible reporting.
- Questions
- 12
- Time
- 30 min
- Pass mark
- 70%
- Retake after
- 6 h
Skills covered
Estimate a standard error by bootstrapping · Build and interpret bootstrap intervals · Recognise when the bootstrap fails · Explain the logic of a permutation test · Compute exact and Monte Carlo permutation p-values · Choose a permutation scheme that matches the design · Estimate probabilities and check methods by simulation · Quantify Monte Carlo error · Design and report a reproducible simulation · Compute robust measures of centre and spread · Reason about breakdown points and resistance · Decide how to treat extreme values · Classify missing-data mechanisms · Explain complete-case bias · Recognise the limits of simple imputation · Vary defensible analysis choices · Find the tipping point of an assumption · Judge whether a conclusion is robust · Compute stratified and standardised rates · Explain Simpson's paradox through confounding · Choose which variables to adjust for · Recognise and quantify survivorship bias · Identify forms of selection bias · Explain spurious associations created by selection · Control false positives across many tests · Recognise p-hacking and forking paths · Explain publication bias and effect inflation · Make an analysis reproducible · Report results honestly and in proportion · Protect respondents' privacy in data releases
Covers Track 13: Resampling and Statistical Reasoning. Passing it, together with completing every lesson, earns the track certificate.
How it works
- Questions are drawn from this assessment's own bank, with numbers generated for your attempt.
- Your answers save automatically. Nothing is graded until you submit or the time runs out (the server keeps the time).
- No hints or feedback during the attempt; full worked solutions afterwards.
- You may use a calculator, the lab and your notes. The assessment is not proctored.