Generative model

A generative model is a model for randomly generating observed data, typically given some hidden parameters. It defines a joint probability distribution over observation and label sequences. Generative models are used in machine learning for either modeling data directly (i.e., modeling observed draws from a probability density function), or as an intermediate step to forming a conditional probability density function. A conditional distribution can be formed from a generative model through the use of Bayes' rule.

Generative models contrast with discriminative models, in that all the variables of a descriptive model are directly measurable.

Examples of generative models include:
 * Gaussian distribution
 * Gaussian mixture model
 * Multinomial distribution
 * Hidden Markov model
 * Generative grammar
 * Naive Bayes

If the observed data are truly generated by the generative model, then fitting the parameters of the generative model to maximize the data likelihood is a common method. However, data rarely truly arises from the generative models used. Therefore, it is often more accurate to model the conditional density functions directly: i.e., performing classification or regression analysis.