Track 16 · Advanced Foundations · Lesson 8 of 8
Model limitations and honest reporting
Identify the limitations of a statistical model — extrapolation, overfitting, violated assumptions, data drift and confounding — and report results with honest uncertainty and scope.
A building manager used last spring's model of energy use against temperature to budget for summer — and the July bills came in far above the forecast. The model was not wrong about spring; it was asked a question its data could not answer.
“All models are wrong, but some are useful” is a reminder, not an excuse. A model is a simplification fitted to particular data, under particular assumptions, for a particular purpose. Its usefulness ends where those conditions end. This lesson collects the usual ways models fail and how to report them honestly.
Included with a Formula Quest membership
It has 8 graded exercises with feedback, hints and worked solutions. After it you will be able to:
- Recognise extrapolation, overfitting, violated assumptions, drift and confounding
- Check the data's range before trusting a prediction
- Explain why a model's metrics change when the population changes
- Choose between confidence and prediction intervals when reporting
- Write a limitations paragraph that states scope, uncertainty, assumptions and design