Track 13 · Resampling and Statistical Reasoning · Lesson 8 of 10
Selection and survivorship bias
Recognise how the way units enter a dataset — surviving, volunteering, being selected or staying in a study — can bias estimates and create spurious relationships, and quantify simple cases.
Fund databases show that the average fund beat the market. But funds that did badly were closed or merged and quietly disappeared from the database. The average of the survivors is an average of winners — it says little about what a new investor should expect.
Selection bias arises when the units in the data are not a fair sample of the units the question is about, because something related to the outcome decided who got in. It cannot be fixed by a larger sample: more of the wrong units gives a more precise wrong answer.
The most common forms:
- Survivorship bias: only the units that survived a process are visible (surviving funds, companies still in business, buildings still standing).
- Self-selection: units choose whether to take part (volunteers, people who reply to reviews).
- Attrition: units drop out of a study during follow-up, and those who leave differ from those who stay.
- Selection on the outcome or on a common effect (Berkson's or collider bias): only units that passed a filter depending on two traits are observed, which creates a link between the traits.
Included with a Formula Quest membership
It has 7 graded exercises with feedback, hints and worked solutions. After it you will be able to:
- Recognise survivorship, self-selection, attrition and selection on a common effect
- Quantify survivorship bias and the gap between completers and everyone who started
- Explain how selecting on a combination of traits creates spurious correlations
- Ask, before generalising, how units came to be in the data and who is missing