Confusion matrix and classification metrics

A confusion matrix counts correct and incorrect predictions at a threshold, and metrics such as precision, recall and specificity summarise different kinds of error.

\text{accuracy} = \frac{TP+TN}{n},\quad \text{precision} = \frac{TP}{TP+FP},\quad \text{recall} = \frac{TP}{TP+FN},\quad \text{specificity} = \frac{TN}{TN+FP}

Core metrics (a case is predicted positive when its score is at least the threshold)

$TP,\ FN$: actual positives predicted positive / negative
$TN,\ FP$: actual negatives predicted negative / positive
$n$: total number of cases

F_1 = \frac{2\cdot\text{precision}\cdot\text{recall}}{\text{precision}+\text{recall}},\qquad \text{balanced accuracy} = \frac{\text{sensitivity}+\text{specificity}}{2}

Combined metrics

$\text{sensitivity}$: the same as recall (true positive rate)

When to use

Evaluating a classifier (logistic regression, a scoring rule, a screening test) at a chosen threshold — preferably on data not used to fit it.

How to interpret

Recall/sensitivity: share of actual positives caught. Specificity: share of actual negatives correctly cleared.
Precision (PPV): share of predicted positives that are truly positive — depends strongly on prevalence.
Moving the threshold trades recall against specificity; choose it from the costs of each error type.
A metric whose denominator is 0 is reported as missing (null), never as 0 or 1.

Common mistakes

Judging by accuracy on imbalanced data: predicting "negative" for everyone scores 95% accuracy when 5% are positive.
Evaluating on the training data (optimistic) instead of a test set or cross-validation.
Confusing precision with recall, or assuming precision transfers to a population with different prevalence.

Example

Ten cases (4 positive) scored 0.9, 0.8, 0.55, 0.3 (positives) and 0.6, 0.4, 0.35, 0.2, 0.1, 0.05 (negatives); threshold 0.5.

TP = 3, FN = 1, FP = 1, TN = 5.
Accuracy 0.80, precision 0.75, recall 0.75, specificity 0.833, F₁ 0.75, balanced accuracy 0.792.