Kernel regression

The kernel regression is a non-parametrical technique in statistics to estimate the conditional expectation of a random variable. The objective is to find a non-linear relation between a pair of random variables X and Y.

In any nonparametric regression, the conditional expectation of a variable $$Y$$ relative to a variable $$X$$ may be written:

$$\operatorname{E}(Y | X) = m(X)$$

where $$m$$ is a non-parametric function.

Nadaraya-Watson kernel regression
Nadaraya (1964) and Watson (1964) proposed to estimate $$m$$ as a locally weighted average, using a kernel as a weighting function. The Nadaraya-Watson estimator is:

$$ \widehat{m}_h(x)=\frac{n^{-1}\sum_{i=1}^nK_h(x-X_i)Y_i }{n^{-1}\sum_{i=1}^nK_h(x-X_i)} $$

where $$K$$ is a kernel with a bandwidth $$h$$.

Derivation
$$ \operatorname{E}(Y | X) = \int y f(y|x) dy = \int y \frac{f(x,y)}{f(x)} dy $$

Using the kernel density estimation for the joint distribution f(x,y) and f(x) with a kernel K,

$$ \hat{f}(x,y) = n^{-1} h^{-2} \sum_{i=1}^{n} K\left(\frac{x-x_i}{h}\right) K\left(\frac{y-y_i}{h}\right) $$,

$$ \hat{f}(x) = n^{-1} h^{-1} \sum_{i=1}^{n} K\left(\frac{x-x_i}{h}\right) $$

we obtain the Nadaraya-Watson estimator.

Priestley-Chao kernel estimator
$$ \widehat{m}_{PC}(x) = h^{-1} \sum_{i=1}^n (x_i - x_{i-1}) K\left(\frac{x-x_i}{h}\right) y_i $$

Gasser-Müller kernel estimator
$$ \widehat{m}_{GM}(x) = h^{-1} \sum_{i=1}^n \left[\int_{s_{i-1}}^{s_i} K\left(\frac{x-u}{h}\right) du\right] y_i $$

where si = (xi-1 + xi)/2

Kernel regression for image processing
Applications of Kernel regression for image processing purposes include but is not limited to denoising, deblurring, interpolation, super-resolution, and many other applications.

Statistical implementation
kernreg2 y x, bwidth(.5) kercode(3) npoint(500) gen(kernelprediction gridofpoints)
 * Stata [kernreg2]