Regularization (machine learning)


 * For other uses in related fields, see Regularization

In machine learning, regularization is any method of preventing overfitting of data by a model. It is used for solving ill-conditioned parameter-estimation problems. Typical examples of regularization in statistical machine learning include ridge regression, lasso, and L2-norm in support vector machines.

Regularization methods are also used for model selection, where they work by implicitly or explicitly penalizing models based on the number of their parameters. Bayesian learning methods, for example, make use of a prior that (generally) gives lower probability to more complex models. Well-known model selection techniques include the Akaike information criterion (AIC), minimum description length (MDL), and the Bayesian information criterion (BIC). Alternative methods of controlling overfitting include cross validation.