Law of large numbers: Meaning, Criticisms & Real-World Uses

What Is the Law of Large Numbers?

The Law of Large Numbers (LLN) is a fundamental theorem in probability theory and statistics that states that as the sample size of independent, identically distributed random events increases, the average of the observed results will converge to the true expected value of the population. This principle suggests that while individual outcomes of a random event may be unpredictable, the average outcome over many trials becomes highly predictable. The Law of Large Numbers is a cornerstone for understanding and applying statistical inference in various fields, including financial mathematics.

History and Origin

The concept behind the Law of Large Numbers has roots in the observations of early mathematicians studying games of chance. The Italian mathematician Gerolamo Cardano noted in the 16th century that empirical statistics tend to improve with more trials, though he provided no formal proof. The first rigorous proof of a special form of the Law of Large Numbers for binary random variables was provided by Jacob Bernoulli. He spent over two decades developing his proof, which was eventually published posthumously in 1713 in his seminal work, Ars Conjectandi (The Art of Conjecturing). Bernoulli referred to this significant discovery as his "golden theorem." Over a century later, in 1837, Siméon Denis Poisson further generalized the theorem and popularized the term "la loi des grands nombres," or "the law of large numbers".¹⁴

Key Takeaways

The Law of Large Numbers posits that the average result of a large number of independent trials will approach the expected value.
It ensures that while short-term results may vary significantly, long-term averages will stabilize and become more predictable.
This principle is crucial for fields like actuarial science, where predicting future events for large groups is essential.
¹³* The Law of Large Numbers does not imply that past deviations from the expected value will be "balanced out" by future outcomes.
It forms a theoretical basis for practices like portfolio diversification and risk assessment in finance.
¹²

Formula and Calculation

The Law of Large Numbers is a theorem rather than a formula that produces a specific numerical output. However, it describes the behavior of a sample mean as the number of observations increases.
For a sequence of independent and identically distributed (i.i.d.) random variables (X_1, X_2, ..., X_n), each with an expected value (E[X_i] = \mu), the sample mean (\bar{X}_n) converges to (\mu) as (n) approaches infinity.

The sample mean is calculated as:

\bar{X}_n = \frac{1}{n} \sum_{i=1}^{n} X_i

The Law of Large Numbers states that:

\lim_{n \to \infty} P(|\bar{X}_n - \mu| < \epsilon) = 1

where:

(\bar{X}_n) is the sample mean of (n) observations.
(\mu) is the true expected value (population mean).
(\epsilon) (epsilon) is any small positive number, representing the allowable deviation from the true mean.
(P) denotes probability.

This mathematical expression indicates that the probability of the sample mean being arbitrarily close to the true expected value approaches 1 as the number of trials (n) becomes very large.

Interpreting the Law of Large Numbers

Interpreting the Law of Large Numbers involves understanding that randomness tends to "average out" over time or with a large number of repetitions. It highlights that the aggregate behavior of many random events is more predictable than the behavior of any single event. For example, while the outcome of a single coin flip is uncertain (either heads or tails), if you flip a fair coin a thousand times, the proportion of heads will likely be very close to 50%. The more times you flip it, the closer that proportion will get to 50%. This doesn't mean that a streak of heads will be "balanced" by an equal streak of tails; rather, the sheer volume of trials dilutes the impact of any short-term fluctuations. This principle is fundamental for statistical inference, allowing reliable conclusions to be drawn from large datasets. It underpins techniques such as regression analysis in financial modeling.
¹¹

Hypothetical Example

Consider an investment strategy based on rolling a fair six-sided die, where each roll represents the annual return of a hypothetical asset. The possible outcomes are 1%, 2%, 3%, 4%, 5%, or 6%, each with a probability theory of 1/6. The expected value (average return) of a single roll is 3.5% (\left( \frac{1+2+3+4+5+6}{6} = 3.5 \right)).

If an investor rolls the die only a few times, say three times, the average return could be far from 3.5%. For example, rolling 6%, 6%, and 3% yields an average return of 5%. However, if the investor repeats this experiment for 1,000 "years" (1,000 rolls of the die), the average annual return observed over those 1,000 rolls would be very close to 3.5%. This is the Law of Large Numbers in action. The individual outcomes remain random, but the cumulative average converges to the true expected value, providing a more stable and predictable long-term result for the overall "portfolio." This concept is a core idea behind understanding expected value in long-term financial planning.

Practical Applications

The Law of Large Numbers has numerous practical applications across finance and economics:

Insurance: Insurance companies heavily rely on the Law of Large Numbers. By insuring a large number of policyholders, insurers can accurately predict the total number of claims and the average cost of those claims within a given period. While they cannot predict which specific individual will file a claim, the large pool of insureds allows them to set profitable insurance premiums and manage their underwriting risk effectively. ¹⁰This principle enables the insurance industry to operate sustainably by ensuring that actual losses converge to expected losses over a broad base.
⁹* Gambling: Casinos and lotteries operate profitably because the Law of Large Numbers guarantees that, over a vast number of bets, the actual outcomes will converge to the statistically determined house edge. While individual gamblers may experience winning streaks, the casino's earnings will tend towards a predictable percentage over many plays.
Portfolio Management: The principle supports portfolio diversification. By combining many different assets, the idiosyncratic risks of individual assets tend to cancel out, leading the overall portfolio's return to converge toward its expected long-term average, reducing overall volatility.
⁸* Financial Modeling and Econometrics: In economic models and financial analysis, the Law of Large Numbers validates the use of empirical estimates. As the volume of financial data (e.g., stock returns, economic indicators) increases, sample averages become more reliable estimates of true population parameters, improving the accuracy of forecasts and statistical inferences.
⁷

Limitations and Criticisms

While powerful, the Law of Large Numbers has important limitations and is often subject to misinterpretation. One common misconception is that it implies a "balancing" effect, where past deviations from the expected average must be corrected by future outcomes. This flawed belief is known as the Gambler's Fallacy. For instance, if a fair coin lands on heads several times in a row, the Gambler's Fallacy incorrectly suggests that tails is "due" to appear to balance the streak. However, each coin flip is an independent event, and the probability of landing on tails remains 50% regardless of previous outcomes. The Law of Large Numbers simply states that over an extremely long series of trials, the proportion of heads and tails will approach 50%.⁶ It does not guarantee that any finite sequence will perfectly reflect the expected probabilities, nor does it predict short-term outcomes.⁵

Another common misapplication, particularly in the financial world, is the "law of large numbers" applied to corporate growth. In this context, it suggests that as a company's market capitalization or revenue growth reaches a very large scale, it becomes increasingly difficult to maintain its previous high growth rates indefinitely.⁴ While this observation about business growth trends may be true due to factors like market saturation or increased competition, it is an economic concept related to diseconomies of scale and should not be confused with the statistical theorem of the Law of Large Numbers.

Law of Large Numbers vs. Gambler's Fallacy

The Law of Large Numbers and the Gambler's Fallacy are often confused, despite being contradictory in their implications for individual events. The Law of Large Numbers is a statistical theorem describing the long-term behavior of averages in sequences of independent trials: as the number of trials increases, the observed average of outcomes will converge towards the true expected value. It is a mathematical truth about cumulative results.

In contrast, the Gambler's Fallacy is a cognitive bias, an erroneous belief that past outcomes of random events influence future outcomes. For example, after a roulette wheel lands on black multiple times, a person exhibiting the Gambler's Fallacy might believe that red is more likely to appear on the next spin to "balance out" the results. This is incorrect because each spin of the roulette wheel is an independent event; the probability of red or black remains constant for every spin, irrespective of prior outcomes. The Law of Large Numbers does not predict or guarantee that short-term fluctuations will be immediately corrected; it only speaks to the overall stability of averages over a very large number of trials.
², ³

FAQs

Q: Does the Law of Large Numbers guarantee a specific outcome in the short term?
A: No. The Law of Large Numbers only applies to the average of results over a very large number of trials. It does not guarantee any specific outcome or "balancing" in the short term. Individual events remain unpredictable.

Q: How does the Law of Large Numbers apply to investing?
A: In investing, the Law of Large Numbers supports the concept of portfolio diversification. By investing in a large number of uncorrelated assets, the unique, unpredictable risks of individual assets tend to cancel each other out over time, leading the overall portfolio's returns to more closely align with its expected long-term average return.

Q: Is the Law of Large Numbers related to the "law of averages"?
A: The "law of averages" is a colloquial term that often refers to the Law of Large Numbers, but it is frequently misused to imply that past events influence future random outcomes (similar to the Gambler's Fallacy). The proper statistical understanding is that averages stabilize over a sufficiently large sample size, not that individual results will "even out."

Q: Can the Law of Large Numbers be used to predict individual stock movements?
A: No, the Law of Large Numbers cannot predict the movement of individual stocks. Stock prices are influenced by numerous complex and often unpredictable factors. The theorem applies to the aggregate behavior of many independent random events, not single, non-independent occurrences in financial markets.

Q: What is the difference between the Weak Law of Large Numbers and the Strong Law of Large Numbers?
A: Both are versions of the Law of Large Numbers. The Weak Law of Large Numbers states that the sample average converges in probability to the expected value. The Strong Law of Large Numbers asserts a more rigorous condition, stating that the sample average converges "almost surely" to the expected value, meaning the probability of convergence is 1. For most practical applications in finance, the implications are similar.¹