The law of large numbers

Law of large numbers (LLN). If $X_1, X_2, \dots$ are independent observations from the same distribution with a finite mean $\mu$, then the running average $\bar{x}_n = (X_1 + \dots + X_n)/n$ gets closer and closer to $\mu$ as $n$ grows: for any small tolerance, the chance that $\bar{x}_n$ is further than that from $\mu$ goes to zero. (The finite-mean condition matters: for an extremely heavy-tailed distribution such as the Cauchy, which has no mean, the running average never settles.)

The same holds for proportions: the proportion of heads in $n$ fair tosses approaches 0.5. This is why a long-run relative frequency can serve as a probability, and why large samples estimate population means well.