Hoeffding's inequality

Hoeffding's inequality, named after Wassily Hoeffding, is a result in probability theory that gives an upper bound on the probability for the sum of random variables to deviate from its expected value.

Let


 * $$X_1, \dots, X_n \!$$

be independent random variables. Assume that the $$X_i$$ are almost surely bounded; that is, assume for $$1\leq i\leq n$$ that


 * $$\Pr(X_i \in [a_i, b_i]) = 1. \!$$

Then, for the sum of these variables


 * $$S = X_1 + \cdots + X_n \!$$

we have the inequality (Hoeffding 1963, Theorem 2):


 * $$\Pr(S - \mathrm{E}[S] \geq nt) \leq \exp \left( - \frac{2\,n^2\,t^2}{\sum_{i=1}^n (b_i - a_i)^2} \right),\!$$

which is valid for positive values of t (where $$\mathrm{E}[S]$$ is the expected value of $$S$$).

This inequality is a special case of the more general Bernstein inequality in probability theory, proved by Sergei Bernstein in 1923. It is also a special case of McDiarmid's inequality.