Track 7 · Estimation and Confidence Intervals · Lesson 1 of 10

Point estimates and their properties

Use sample statistics as point estimates of population parameters and judge an estimator by its bias and its variability.

25 minIntermediate7 graded exercisesFree

A battery maker cannot test every battery it will ever produce, so it tests 40 and reports their mean life as an estimate for the whole production. How good is that single number? It depends on whether the method aims at the right target and how much it wobbles from sample to sample.

A parameter is a fixed number describing a population: the mean battery life $\mu$ of all batteries, the proportion $p$ of all customers who would buy, the population standard deviation $\sigma$ . We almost never know it.

A statistic is a number computed from a sample. When we use a statistic to guess a parameter, the rule is called an estimator and the number it produces for our particular sample is the estimate. Because it is a single number, it is a point estimate.

Parameter (population)	Estimator (sample)	Formula
Mean $\mu$	Sample mean $\bar{x}$	$\frac{1}{n}\sum x_i$
Proportion $p$	Sample proportion $\hat{p}$	$x/n$ (successes ÷ trials)
Variance $\sigma^2$	Sample variance $s^2$	$\frac{1}{n-1}\sum (x_i - \bar{x})^2$
Difference $\mu_1 - \mu_2$	$\bar{x}_1 - \bar{x}_2$	difference of the two sample means

Common parameters and their usual estimators

A different sample would give a different estimate. The estimator therefore has a sampling distribution (see Track 6, Sampling and Sampling Distributions), and we judge the estimator — the method — by two properties of that distribution:

Bias: the distance between the centre of the sampling distribution and the parameter. An estimator is unbiased when, averaged over all possible samples, it equals the parameter exactly.
Variability: how spread out the sampling distribution is. Its standard deviation is called the standard error (SE).

\operatorname{Bias}(\hat\theta) = \E[\hat\theta] - \theta, \qquad \operatorname{MSE}(\hat\theta) = \E\big[(\hat\theta - \theta)^2\big] = \operatorname{Bias}^2 + \SE^2

Bias and mean squared error of an estimator

$\theta$: the parameter (a fixed, unknown number)
$\hat\theta$: the estimator (a random quantity: it changes from sample to sample)
$\E[\hat\theta]$: the average of the estimator over all possible samples
$\SE$: the standard error: the standard deviation of the estimator's sampling distribution

The mean squared error (MSE) combines both properties: the average squared miss. Sometimes a slightly biased estimator with a much smaller SE has a lower MSE than an unbiased but noisy one.

For the sample mean, the standard error has a simple form. It shrinks with the square root of the sample size, so quadrupling $n$ halves the SE.

\SE(\bar{x}) = \frac{\sigma}{\sqrt{n}} \approx \frac{s}{\sqrt{n}}, \qquad \SE(\hat{p}) = \sqrt{\frac{p(1-p)}{n}}

Standard errors of the sample mean and the sample proportion

$\sigma$: the population standard deviation (estimated by the sample SD $s$ when unknown)
$n$: the sample size
$p$: the population proportion (estimated by $\hat{p}$ when unknown)

Worked example

Why the sample variance divides by n − 1

Take the tiny population {2, 4, 9}, whose variance (dividing by $N = 3$ ) is $\sigma^2 = 8.667$ . List all 9 samples of size 2 drawn with replacement and average two versions of the sample variance over them.

The 9 equally likely samples are (2, 2), (2, 4), (2, 9), (4, 2), (4, 4), (4, 9), (9, 2), (9, 4), (9, 9).
Dividing by $n - 1 = 1$ , the sample variances average to 8.667 — exactly $\sigma^2$ . The estimator $s^2$ is unbiased.
Dividing by $n = 2$ instead, they average to 4.333, only $(n-1)/n = 1/2$ of $\sigma^2$ . Deviations from $\bar{x}$ are systematically too small, because $\bar{x}$ is computed from the same values and sits closer to them than $\mu$ does.
Taking square roots does not keep unbiasedness: the sample SDs $s$ average to 2.200, below $\sigma = 2.944$ . So $s^2$ is unbiased for $\sigma^2$ , but $s$ slightly underestimates $\sigma$ on average (the bias fades as $n$ grows).

Dividing by n − 1 makes the sample variance an unbiased estimator of the population variance.

Draw many samples and look at the sampling distribution of an estimator.

Try this

With the mean, note the centre of the simulated sample means and their SD (the simulated standard error). Both estimate properties of the estimator, not of one sample.
Switch the statistic to the median. It is also centred on the population centre for a symmetric population, but its spread is larger: for normal data the mean is the more precise estimator.
Change $n$ from 10 to 40. By roughly what factor does the standard error of the mean shrink? (Theory says 2, because $\sqrt{4} = 2$ .)

Quick check

An estimator is unbiased. What does that guarantee?

Exercises

The lesson is complete when every exercise is answered correctly. Hints are free; viewing a worked solution before solving means the exercise won't count towards mastery.

0 of 7 exercises correct (not saved — sign in to keep your progress)

Exercise 1 · Guided, difficulty 1 of 3

Estimate the mean and the standard deviation

Not started

A quality engineer tests 10 batteries from a new production line. Their lives (hours) are:

48.1, 42.3, 52.6, 59.1, 46.5, 49.3, 50.4, 57.2, 35.4, 49.7

Give the point estimates of the mean life $\mu$ and of the standard deviation $\sigma$ of all batteries from the line (2 decimal places).

Exercise 2 · Practice, difficulty 1 of 3

Parameter or statistic?

Not started

Sort each quantity: is it a parameter (a fixed number describing a whole population, usually unknown) or a statistic (computed from a sample, usable as an estimate)?

The proportion of all registered voters in a city who support a new park

The proportion supporting the park among 800 voters phoned by a polling firm

The long-run mean fill weight of every bottle a machine produces at its current setting

The mean weight of 50 bottles taken from today's production

The sample standard deviation of 30 measured delivery times

The true difference in mean blood pressure between treated and untreated patients in the target population

Exercise 3 · Practice, difficulty 1 of 3

Standard error of a mean

Not started

In an online survey, the time respondents spend on one question is recorded. A sample of $n = 25$ respondents has a standard deviation of $s = 15.1$ seconds. Estimate the standard error of the sample mean time (3 decimal places).

Exercise 4 · Challenge, difficulty 3 of 3

Biased but better?

Not started

Two methods estimate the same parameter. Over all possible samples:

Estimator A is unbiased with standard error 3.
Estimator B has bias 2 and standard error 2.

Compute the mean squared error of each (2 decimal places). Which would you rather use?

Exercise 5 · Application, difficulty 2 of 3

Estimate the mean commute and its precision

Not started

Treat the 240 employees in the commute dataset as a random sample of office workers in the same business district (an assumption you would need to justify in practice). Estimate the population mean commute time $\mu$ from commute_min, and the standard error of that estimate (2 decimal places for the mean, 3 for the SE).

Data: commute · Open in the lab to explore and calculate.

Exercise 6 · Practice, difficulty 2 of 3

The standard error of a proportion

Not started

Write the standard error of the sample proportion $\hat{p}$ in terms of the population proportion p and the sample size n. For example, sqrt(p/n) is a valid format (but not the right answer).

Exercise 7 · Interpretation, difficulty 2 of 3

Is the sample SD unbiased?

Not started

Which statement about the sample variance $s^2$ (divisor $n - 1$ ) and the sample standard deviation $s$ is correct?

What you can now do

Distinguish a parameter, an estimator and an estimate
Describe an estimator by its bias and its standard error, and combine them in the MSE
Calculate the standard error of a mean and of a proportion
Explain why the sample variance divides by n − 1