Track 13 · Resampling and Statistical Reasoning · Lesson 6 of 10
Sensitivity analysis
Test whether a conclusion survives other defensible analysis choices and assumptions, find tipping points, and report fragile results honestly.
An engagement report announces that “a majority of staff are proud to work here”: 50.5% of respondents agreed. Remove 18 obviously careless responses, or weight departments by their size, and the majority disappears. A conclusion that depends on a choice nobody would defend as uniquely right is not a finding — it is a coin toss.
Every analysis rests on choices: which records to exclude, how to treat outliers and missing values, which summary or model to use, whether to weight. Usually several choices are defensible. A sensitivity analysis repeats the analysis under the main alternatives and asks: does the conclusion change?
- If it does not, the conclusion is robust and you can say so with more confidence.
- If it does, the conclusion is fragile: report the range, explain which choice drives it, and do not present one version as the answer.
Included with a Formula Quest membership
It has 8 graded exercises with feedback, hints and worked solutions. After it you will be able to:
- Re-run an analysis under the defensible alternative choices and tabulate the results
- Distinguish a robust estimate from a fragile headline that sits on a threshold
- Compute a tipping point for an untestable assumption such as nonresponse
- Separate sensitivity checks from analyses that answer a different question