ROC curve and AUC

The ROC curve plots the true positive rate against the false positive rate over all thresholds, and the AUC is the probability that a random positive outscores a random negative.

\text{TPR}(t) = \frac{TP(t)}{P},\qquad \text{FPR}(t) = \frac{FP(t)}{N} = 1 - \text{specificity}(t)

ROC coordinates at threshold t (score ≥ t predicted positive)

$P,\ N$: numbers of actual positives and negatives
$t$: threshold; each distinct score gives one point

\text{AUC} = P(S_+ > S_-) + \tfrac12 P(S_+ = S_-)

Area under the ROC curve (trapezoidal area)

$S_+,\ S_-$: scores of a randomly chosen positive and negative case

When to use

Comparing how well scores rank positives above negatives, independently of any single threshold.

How to interpret

AUC = 0.5: no better than random ranking; 1: perfect separation; below 0.5: the scores point the wrong way.
Each point on the curve is a possible operating threshold; pick one from the costs of false positives and false negatives.
AUC equals the Mann–Whitney U statistic of the scores divided by $P \times N$ .

Common mistakes

Treating a high AUC as proof of well-calibrated probabilities; AUC ignores calibration.
Using AUC alone when only a narrow range of false positive rates is acceptable in practice.
Computing AUC on the training data.

Example

Same ten cases as in classification metrics. Of the 4 × 6 = 24 positive–negative pairs, the positive scores higher in 20, so AUC = 20/24 = 0.833. The threshold 0.55 gives the point (FPR 0.167, TPR 0.75).