The bootstrap: standard errors and intervals

Resample the data with replacement many times, recompute the statistic each time, and use the spread of those replicates to estimate its standard error and a confidence interval (percentile or BCa).

\widehat{\SE}_{\text{boot}} = \sqrt{\frac{1}{B-1}\sum_{b=1}^{B}\left(\hat\theta^*_b - \bar\theta^*\right)^2}, \qquad \text{percentile CI} = \left[q_{\alpha/2}, \; q_{1-\alpha/2}\right]

Bootstrap SE and percentile interval

$\hat\theta^*_b$: statistic computed on bootstrap resample $b$ (n draws with replacement)
$B$: number of resamples (the engine defaults to 10 000)
$q_p$: type 7 quantile of the $B$ replicates at probability $p$

z_0 = \Phi^{-1}\!\left(\frac{\#\{\hat\theta^*_b < \hat\theta\}}{B}\right), \quad a = \frac{\sum(\bar\theta_{(\cdot)} - \theta_{(i)})^3}{6\left[\sum(\bar\theta_{(\cdot)} - \theta_{(i)})^2\right]^{3/2}}, \quad \alpha' = \Phi\!\left(z_0 + \frac{z_0 + z_\alpha}{1 - a(z_0 + z_\alpha)}\right)

BCa interval: replicate quantiles at adjusted levels

$z_0$: bias correction: how far the replicates sit below the original estimate $hat heta$
$a$: acceleration, from the jackknife (leave-one-out) values $heta_{(i)}$ and their mean $ar heta_{(cdot)}$
$\alpha'$: adjusted probability at which the replicate quantile is read, for each end

When to use

Standard errors and intervals for statistics without a simple formula (median, trimmed mean, ratio, correlation, percentile) or when the normal approximation is doubtful.
Prefer BCa over the plain percentile interval for skewed or biased statistics.

Assumptions

The observations are an independent random sample, so the sample stands in for the population (resample whole units; for time series or clusters use block or cluster resampling).
n is not tiny: with small samples every bootstrap interval tends to be too narrow.
Statistics that depend on a few extreme values (maximum, minimum) are poorly bootstrapped.

How to interpret

The bootstrap distribution shows how much the statistic would vary across samples; it is not a picture of the population.
A 95% bootstrap interval is a confidence interval with approximately (not exactly) 95% coverage.
Results depend slightly on the random seed; record the seed and $B$ so the analysis can be reproduced.

Common mistakes

Thinking resampling creates new information or fixes a biased sample.
Resampling individual observations when the data are paired, clustered or serially dependent.
Using too few resamples (hundreds) for interval end points.

Example

Sample 12, 15, 9, 20, 14, 11, 30, 13; median = 13.5. With $B$ = 10 000 and seed 42: bootstrap SE = 2.04; 95% percentile interval [11, 20]; BCa interval [11, 17.5] ( $z_0 = -0.18$ ). The median of only 8 values has a lumpy bootstrap distribution, so interval ends land on data values or midpoints.