Track 16 · Advanced Foundations · Lesson 1 of 8

Bayesian thinking: prior, likelihood, posterior

Combine a prior distribution with the likelihood of observed data to obtain a posterior distribution, and explain how the prior and the data share the influence.

35 minAdvanced8 graded exercisesFree

A quality engineer already suspects that a new supplier is probably fine, but after 40 parts with 4 defects her suspicion should change — by how much? Bayesian inference turns “what I believed” plus “what I saw” into “what I should now believe”, with the arithmetic made explicit.

In the methods of the earlier tracks, a parameter such as a defect rate $\theta$ is a fixed unknown number and probability describes how data vary from sample to sample. Bayesian inference also uses probability to describe what we know about the parameter. Before the data, that knowledge is the prior distribution. The data enter through the likelihood: how probable the observed data would be for each possible value of $\theta$ . Bayes' rule combines them into the posterior distribution: what we know about $\theta$ after seeing the data.

p(\theta \given \text{data}) = \frac{p(\text{data} \given \theta)\, p(\theta)}{p(\text{data})} \;\propto\; \underbrace{p(\text{data} \given \theta)}_{\text{likelihood}} \times \underbrace{p(\theta)}_{\text{prior}}

Bayes' rule for a parameter: the posterior is proportional to likelihood × prior

$p(\theta)$: the prior: plausibility of each value of $\theta$ before seeing the data
$p(\text{data} \given \theta)$: the likelihood: probability of the observed data if $\theta$ were the true value
$p(\theta \given \text{data})$: the posterior: plausibility of each value of $\theta$ after seeing the data
$p(\text{data})$: the total probability of the data, summed over all values of $\theta$ ; it only rescales the products so the posterior adds to 1

Worked example

Is the new supplier good, average or poor?

From experience with similar suppliers, an engineer gives prior probabilities 0.5 to “good” (defect rate 2%), 0.3 to “average” (defect rate 5%), 0.2 to “poor” (defect rate 10%). She inspects 40 parts and finds 4 defective. What are the posterior probabilities?

Likelihood of the data under each state (binomial, $n = 40$ , $k = 4$ ): $P(4 \given 0.02) = 0.0071$ ; $P(4 \given 0.05) = 0.0901$ ; $P(4 \given 0.1) = 0.2059$ .
Multiply by the prior: good $0.5 \times 0.0071 = 0.00353$ ; average $0.3 \times 0.0901 = 0.02704$ ; poor $0.2 \times 0.2059 = 0.04118$ .
Normalise by the total $0.07175$ so the posterior adds to 1: good 0.049, average 0.377, poor 0.574.
Compare with the prior. “Good” fell from 0.5 to 0.05 and “poor” rose from 0.2 to 0.57: an observed rate of 4/40 = 10% is far more probable for an average or poor supplier.

Posterior: good 0.049, average 0.377, poor 0.574. With equal prior probabilities (1/3 each) the posterior would have been 0.023, 0.297, 0.679 — the prior still matters with only 40 parts.

State	Defect rate θ	Prior	Likelihood	Prior × likelihood	Posterior
good	0.02	0.5	0.0071	0.00353	0.049
average	0.05	0.3	0.0901	0.02704	0.377
poor	0.1	0.2	0.2059	0.04118	0.574

The same calculation as a table: prior × likelihood, then rescale

You already know one famous Bayesian update: a diagnostic test (see Track 4, Probability). The prevalence is the prior probability of the condition; the test's sensitivity and specificity give the likelihood of a positive result; the positive predictive value is the posterior. A second, independent positive test then starts from that posterior as its prior.

The same logic works for a continuous parameter. Suppose a delivery route's mean time $\mu$ has a normal prior with mean $\mu_0$ and standard deviation $\tau_0$ , and individual deliveries vary around $\mu$ with a known standard deviation $\sigma$ . After $n$ deliveries with mean $\bar{x}$ , the posterior for $\mu$ is again normal, and its mean is a precision-weighted average of the prior mean and the data mean (precision = 1 / variance).

\mu_n = w\,\bar{x} + (1 - w)\,\mu_0,\qquad w = \frac{n/\sigma^2}{1/\tau_0^2 + n/\sigma^2},\qquad \tau_n = \sqrt{\frac{1}{1/\tau_0^2 + n/\sigma^2}}

Normal prior, normal data with known σ: the posterior mean is a compromise

$\mu_0, \tau_0$: prior mean and prior standard deviation of $\mu$
$\bar{x}, n, \sigma$: sample mean, sample size, and the known SD of individual observations
$w$: the weight on the data: it grows towards 1 as $n$ grows
$\mu_n, \tau_n$: posterior mean and posterior standard deviation of $\mu$

With prior mean 42 minutes (SD 3), $\sigma = 9$ and a sample mean of 50 minutes: from 4 deliveries the data weight is $w = 0.308$ and the posterior mean is 44.46 minutes (SD 2.50); from 100 deliveries $w = 0.917$ and the posterior mean is 49.34 minutes (SD 0.86). A little data barely moves a confident prior; a lot of data overwhelms it.

Quick check

After the Bayesian update above, the posterior probability that the supplier is “poor” is 0.57. What does 0.57 describe?

Exercises

The lesson is complete when every exercise is answered correctly. Hints are free; viewing a worked solution before solving means the exercise won't count towards mastery.

0 of 8 exercises correct (not saved — sign in to keep your progress)

Exercise 1 · Guided, difficulty 1 of 3

Update a three-state prior

Not started

Prior probabilities for a new supplier's state are good (defect rate 2%) 0.5, average (5%) 0.3 and poor (10%) 0.2. An inspection of 50 parts finds 3 defective.

Using binomial likelihoods, find the posterior probabilities that the supplier is good and that it is poor (3 decimal places).

Exercise 2 · Practice, difficulty 2 of 3

A second positive test

Not started

A screening test has sensitivity 0.92 and specificity 0.94; the condition affects 3.0% of the people screened. A person tests positive, then takes a second test whose errors are independent of the first, and is positive again.

Give the probability that they have the condition after the first positive and after the second positive (3 decimal places).

Exercise 3 · Practice, difficulty 2 of 3

A normal prior for a mean

Not started

Experience with similar routes gives a normal prior for the mean delivery time $\mu$ of a new route: mean 38 minutes, standard deviation 4 minutes. Individual deliveries vary around $\mu$ with a known SD of $\sigma = 6$ minutes. The first 8 deliveries average $\bar{x} = 47.2$ minutes.

Find the posterior mean and posterior standard deviation of $\mu$ (2 decimal places).

Exercise 4 · Application, difficulty 2 of 3

Prior and posterior in the screening data

Not started

Use the screening dataset, restricted to the 60+ age group (the exercise data are already filtered). Treat the proportion of people with the condition as the prior probability for a person in this group, and the proportion with the condition among those who tested positive as the posterior after a positive test.

Give both as proportions (3 decimal places), and the sensitivity $P(+ \given \text{condition})$ for comparison.

Data: screening (age_group = 60+) · Open in the lab to explore and calculate.

Exercise 5 · Practice, difficulty 1 of 3

Name the ingredient

Not started

A team estimates the click-through rate $\theta$ of a new banner. Classify each quantity as part of the prior, the likelihood or the posterior.

Past banners had click-through rates mostly between 1% and 3%, so

\theta

is probably in that range

The probability of observing 38 clicks in 2 000 views if

\theta

were 2%

After the test, a 90% probability that

\theta

lies between 1.5% and 2.3%

The binomial model

P(k \given \theta) = \binom{n}{k}\theta^k(1-\theta)^{n-k}

P(\theta > 2\% \given 38 \text{ clicks in } 2\,000 \text{ views})

A Beta(2, 98) distribution chosen before the test

Exercise 6 · Practice, difficulty 1 of 3

The Bayesian workflow

Not started

Put the steps of a Bayesian analysis in a sensible order.

Exercise 7 · Interpretation, difficulty 2 of 3

When does the prior matter?

Not started

For a route's mean delivery time, the prior is normal with mean 42 minutes (SD 3) and $\sigma = 9$ . The sample mean is 50 minutes. With $n = 4$ the posterior mean is 44.46; with $n = 100$ it is 49.34. Which statements are correct? (Select all that apply.)

Exercise 8 · Interpretation, difficulty 2 of 3

Explain the update to a manager

Not started

In the supplier example, the prior probability that the supplier is “poor” was 0.2; after 4 defects in 40 parts the posterior is 0.57. A manager asks: “Why isn't it 100% poor, when 10% of the parts were defective — exactly the poor rate?” Answer in two or three sentences.

What you can now do

Explain the roles of the prior, the likelihood and the posterior
Update a discrete prior with binomial data by multiplying and rescaling
Use a posterior as the prior for the next batch of data
Compute a normal–normal posterior as a precision-weighted compromise
Judge when the prior matters and when the data dominate