In probability theory, the law of large numbers (LLN) is a theorem that describes the result of performing the same experiment a large number of times. According to the law, the average of the results obtained from a large number of trials should be close to the expected value, and will tend to become closer as more trials are performed.
For example, a single roll of a six-sided die produces one of the numbers 1, 2, 3, 4, 5, or 6, each with equal probability. Therefore, the expected value of a single die roll is
According to the law of large numbers, if a large number of six-sided dice are rolled, the average of their values (sometimes called the sample mean) is likely to be close to 3.5, with the accuracy increasing as more dice are rolled.
It follows from the law of large numbers that the empirical probability of success in a series of Bernoulli trials will converge to the theoretical probability. For a Bernoulli random variable, the expected value is the theoretical probability of success, and the average of n such variables (assuming they are independent and identically distributed (i.i.d.)) is precisely the relative frequency.
For example, a fair coin toss is a Bernoulli trial. When a fair coin is flipped once, the theoretical probability that the outcome will be heads is equal to 1/2. Therefore, according to the law of large numbers, the proportion of heads in a "large" number of coin flips "should be" roughly 1/2. In particular, the proportion of heads after n flips will almost surely converge to 1/2 as n approaches infinity.
Though the proportion of heads (and tails) approaches 1/2, almost surely the absolute (nominal) difference in the number of heads and tails will become large as the number of flips becomes large. That is, the probability that the absolute difference is a small number, approaches zero as the number of flips becomes large. Also, almost surely the ratio of the absolute difference to the number of flips will approach zero. Intuitively, expected absolute difference grows, but at a slower rate than the number of flips, as the number of flips grows.
The LLN is important because it "guarantees" stable long-term results for random events. For example, while a casino may lose money in a single spin of the roulette wheel, its earnings will tend towards a predictable percentage over a large number of spins. Any winning streak by a player will eventually be overcome by the parameters of the game. It is important to remember that the LLN only applies (as the name indicates) when a large number of observations are considered. There is no principle that a small number of observations will converge to the expected value or that a streak of one value will immediately be "balanced" by the others. See the Gambler's fallacy.
Famous quotes containing the words law of, law, large and/or numbers:
“The very existence of government at all, infers inequality. The citizen who is preferred to office becomes the superior to those who are not, so long as he is the repository of power, and the child inherits the wealth of the parent as a controlling law of society.”
—James Fenimore Cooper (1789–1851)
“The law of nature is, do the thing, and you shall have the power: but they who do not the thing have not the power.”
—Ralph Waldo Emerson (1803–1882)
“Frequently also some fair-weather finery ripped off a vessel by a storm near the coast was nailed up against an outhouse. I saw fastened to a shed near the lighthouse a long new sign with the words “ANGLO SAXON” on it in large gilt letters, as if it were a useless part which the ship could afford to lose, or which the sailors had discharged at the same time with the pilot. But it interested somewhat as if it had been a part of the Argo, clipped off in passing through the Symplegades.”
—Henry David Thoreau (1817–1862)
“The land cannot be cleansed of the blood that is shed therein, but by the blood of him that shed it.”
—Bible: Hebrew Numbers 35:33.