Track 1 · Statistical Thinking · Lesson 6 of 10
Where data come from
Classify data sources, distinguish random from systematic measurement error, and check a dataset's documentation and provenance before trusting it.
A planning team downloads a spreadsheet of “clinic visits by district” and uses it to decide where to open a new clinic. Nobody checks that one district's figures exclude walk-in patients, or that the definition of a visit changed halfway through the series. The analysis is flawless; the decision is wrong.
Every dataset was produced by someone, for some purpose, using some procedure. That history — its provenance — limits what the data can tell you.
- Primary data are collected by you for your question: your own survey, experiment or measurements. You control the design but pay the cost.
- Secondary data were collected by someone else, usually for another purpose: government statistics, a company's transaction logs, a published dataset. Cheap and often large, but the definitions and coverage were chosen for their purpose, not yours.
| Source | Example | Strengths | Watch for |
|---|---|---|---|
| Administrative records | Payroll, hospital admissions, tax filings, school enrolment | Complete coverage of whoever is in the system; cheap | Only covers people who interact with the system; definitions follow rules, not research needs |
| Surveys | Customer satisfaction poll, labour-force survey | Can ask exactly what you need; can be random | Nonresponse, question wording, honest-answer problems |
| Designed experiments | A/B test, clinical trial, field trial | Supports causal conclusions | Cost, artificial settings, dropout |
| Sensors and logs | Smart meters, website clickstreams, GPS traces | Huge volume, automatic, time-stamped | Device faults, bots, gaps, calibration drift |
No measurement is perfect. The difference between a recorded value and the true value is measurement error, and it comes in two kinds.
- Random error pushes individual readings up or down unpredictably. It makes data noisier, but it tends to cancel out when you average many readings.
- Systematic error (measurement bias) pushes readings in the same direction every time: a scale that reads 0.4 kg heavy, a survey question that encourages flattering answers, a clock that runs fast. Averaging more readings does not remove it.
Worked example
A miscalibrated scale
A warehouse weighs five parcels on a scale that, unknown to staff, reads 0.4 kg too heavy. The recorded weights are 2.9, 4.1, 3.6, 5.0 and 3.4 kg. What is the mean recorded weight, and what is the true mean?
Mean of the recorded weights: kg.
Every reading is 0.4 kg too high, so the mean is also 0.4 kg too high: true mean kg.
Weighing 5,000 parcels instead of five would make the random wobble in the average smaller — but the average would still be 0.4 kg too high. Only recalibration (or a correction) removes systematic error.
Recorded mean 3.80 kg; true mean 3.40 kg.
Two related words describe measurement quality. A measurement is reliable if repeating it gives similar results (little random error). It is valid if it measures what it claims to measure (no systematic error, and the right concept). A bathroom scale that always reads 2 kg light is reliable but not valid; “number of hours logged in” is a reliable but not necessarily valid measure of productivity.
Provenance questions to ask before using a dataset
- Who collected it, and why? What were they trying to measure or administer?
- What exactly does each variable mean? Is there a data dictionary with units and definitions?
- Who is included and who is missing? How were units selected?
- When and where were the data collected? Did definitions, systems or procedures change during that period?
- How were values measured or recorded — instrument, self-report, manual entry? How are missing values coded?
- What processing happened before you received it (cleaning, imputation, aggregation, rounding)?
Quick check
A fitness tracker counts arm movements as steps, so it consistently overcounts by about 10%. What kind of error is that?
Exercises
The lesson is complete when every exercise is answered correctly. Hints are free; viewing a worked solution before solving means the exercise won't count towards mastery.
0 of 7 exercises correct (not saved — sign in to keep your progress)
Exercise 1 · Practice, difficulty 1 of 3
Primary or secondary?
From the point of view of the analyst described, classify each dataset as primary or secondary data.
Exercise 2 · Practice, difficulty 1 of 3
Name the source
Match each dataset to the type of source.
Exercise 3 · Practice, difficulty 2 of 3
Random or systematic error?
Classify each source of measurement error.
Exercise 4 · Guided, difficulty 1 of 3
Correcting a known bias
A water-quality team measured the chlorine level (mg/L) at six taps:
5.19, 4.05, 4.19, 4.29, 4.96, 4.56
Later it found that the meter reads 0.15 mg/L too high on every measurement. What is the corrected mean chlorine level? Give 3 decimal places.
Exercise 5 · Practice, difficulty 2 of 3
Reliable, valid, both or neither
A company measures “employee productivity” as the number of minutes each employee's computer is active per day. Activity is recorded precisely and consistently by software. Which description fits best?
Exercise 6 · Practice, difficulty 2 of 3
Check before you use
You download a public spreadsheet titled “Road accidents by city, 2018–2024” to compare road safety across cities. Select all the questions you should answer before analysing it.
Exercise 7 · Interpretation, difficulty 3 of 3
Hospital records as a measure of illness
A health department proposes to estimate how common asthma is among adults in a region by counting the adults admitted to the region's hospitals with asthma last year and dividing by the adult population. In two to four sentences, explain the main problems with using these administrative data for this purpose.
What you can now do
- Classify data as primary or secondary and name the type of source
- Distinguish random error (averages out) from systematic error (does not)
- Judge whether a measure is reliable and valid for the concept of interest
- Ask provenance questions about coverage, definitions, changes and processing before analysing