Lindley's paradox

Lindley's paradox describes a counterintuitive situation in statistics in which the Bayesian and frequentist approaches to a hypothesis testing problem give opposite results for certain choices of the prior distribution. The problem of the disagreement between the two approaches was discussed in Harold Jeffreys' textbook ; it became known as Lindley's paradox after Dennis Lindley called the disagreement a paradox in a 1957 paper.

Description of the paradox
Consider a null hypothesis H0, the result of an experiment x, and a prior distribution that favors H0 weakly. Lindley's paradox occurs when
 * 1) The result x is significant by a frequentist test, indicating sufficient evidence to reject H0, say, at the 5% level, and
 * 2) The posterior probability of H0 given x is high, say, 95%, indicating  strong evidence that H0 is in fact true.

These results can happen at the same time when the prior distribution is the sum of a sharp peak at H0 with probability p and a broad distribution with the rest of the probability 1-p. It is a result of the prior having a sharp feature at H0 and no sharp features anywhere else.

Example: heavier-than air flight
To see why this might happen, consider the example of an experiment in a very well-established field of science, such as gravitation, and consider the null hypothesis to be something like, "what we currently believe about gravity is true." The prior probability for this hypothesis is of course very large.

Now, suppose that in this particular experiment, something very unusual happens, and for whatever reason the results of observations appear to be inconsistent with gravitation as we understand it (e.g., a 1,000 ton metal structure lifts off the ground and flies at near the speed of sound). The conditional probability of this occurrence is small given the null hypothesis, and so a frequentist ought to reject it.

Yet a Bayesian knows that the prior probability of the null hypothesis is so high that this experiment is not sufficient to actually conclude that the null hypothesis is false, but rather that something unusual is going on (i.e. powered air flight). In practice, even scientists who consider themselves "frequentist" would adopt this methodology, considering information outside the experiment in drawing their inferences.