Illustration of the central limit theorem

Here is a static illustration of the central limit theorem (you can also see the dynamic SOCR CLT Activity). A probability density function is shown in the first figure. Then the densities of the sums of two, three, and four independent identically distributed variables, each having the original density, are shown in the following figures. Although the original density is far from normal, the density of the sum of just a few variables with that density is much smoother and has some of the qualitative features of the normal density.

A more concrete illustration, in which most of the arithmetic can be done more-or-less instantly by hand, is at concrete illustration of the central limit theorem. There is also a free full-featured interactive simulation available which allows you to set up various distributions and adjust the sampling parameters (see "external links" at the bottom of this page).

Density of a sum of random variables
The density of the sum of two independent real-valued random variables equals the convolution of the density functions of the original variables.

Thus, the density of the sum of m+n terms of a sequence of independent identically distributed variables equals the convolution of the densities of the sums of m terms and of n term. In particular, the density of the sum of n+1 terms equals the convolution of the density of the sum of n terms with the original density (the "sum" of 1 term).

If the original density is a piecewise polynomial, then so are the sum densities, of increasingly higher degree.

The example
The convolutions were computed via the discrete Fourier transform. A list of values y = f(x0 + k &Delta;x) was constructed, where f is the original density function, and &Delta;x is approximately equal to 0.002, and k is equal to 0 through 1000. The discrete Fourier transform Y of y was computed. Then the convolution of f with itself is proportional to the inverse discrete Fourier transform of the pointwise product of Y with itself.



Original density function
We start with a probability density function. This function, although discontinuous, is far from the most pathological example that could be created. It is a piecewise polynomial, with pieces of degrees 0 and 1. The mean of this distribution is 0 and its standard deviation is 1.



Sum of two terms
Next we compute the density of the sum of two independent variables, each having the above density. The density of the sum is the convolution of the above density with itself.

The sum of two variables has mean 0. The density shown in the figure at right has been rescaled by &radic;2, so that its standard deviation is 1.

This density is already smoother than the original. There are obvious lumps, which correspond to the intervals on which the original density was defined.



Sum of three terms
We then compute the density of the sum of three independent variables, each having the above density. The density of the sum is the convolution of the first density with the second.

The sum of three variables has mean 0. The density shown in the figure at right has been rescaled by &radic;3, so that its standard deviation is 1.

This density is even smoother than the preceding one. The lumps can hardly be detected in this figure.



Sum of four terms
Finally, we compute the density of the sum of four independent variables, each having the above density. The density of the sum is the convolution of the first density with the third.

The sum of four variables has mean 0. The density shown in the figure at right has been rescaled by &radic;4, so that its standard deviation is 1.

This density appears qualitatively very similar to a normal density. No lumps can be distinguished by the eye.