Multinomial logit

In statistics and economics, a multinomial logit model is a regression model which generalizes logistic regression to where can be more than two cases.

Introduction
Multinomial logit regression is used when the dependent variable in question is nominal (a set of categories which cannot be ordered in any meaningful way) and consists of more than two categories. For example, multinomial logit regression would be appropriate when trying to determine what factors predict which major college students choose.

Multinomial logit regression is also appropriate in cases where the parallel regression assumption of ordered logit regression is violated. Ordered logit regression is used in cases where the dependent variable in question consists of a set number (more than two) of categories which can be ordered in a meaningful way (for example, highest degree, social class).

Assumptions
The multinomial logit model assumes that data are case specific; that is, each independent variable has a single value for each case. The multinomial logit model also assumes that the dependent variable cannot be perfectly predicted from the independent variables for any case. Collinearity is assumed to be relatively low, as it becomes difficult to differentiate between the impact of several variables if they are highly correlated. The independence of irrelevant alternatives is another assumption which the multinomial logit model makes. This assumption states that the odds do not depend on other alternatives that are available (i.e., that including additional alternatives or deleting alternatives will not affect the odds on the dependent variable among the alternatives that were included originally).

Usage
When using multinomial logistic regression, one category of the dependent variable is chosen as the comparison category. Separate relative risk ratios are determined for all independent variables for each category of the independent variable with the exception of the comparison category of the dependent variable, which is omitted from the analysis. Relative risk ratios, the exponential beta coefficient, represent the change in the odds of being in the dependent variable category versus the comparison category associated with a one unit change on the independent variable.

Model

 * $$\Pr(y_{i}=j)=\frac{\exp(X_{i}\beta_{j})}{1+\sum_{j}^{J}\exp(X_{i}\beta_{j})}$$

and


 * $$\Pr(y_{i}=0)=\frac{1}{1+\sum_{j}^{J}\exp(X_{i}\beta_{j})},$$

where for the ith individual, yi is the observed outcome and Xi is a vector of explanatory variables. The unknown parameters &beta;j are typically estimated by maximum likelihood.

Difference with the conditional logit
While these equations are very similar to the equations for the conditional logit, there is a fundamental difference: for the conditional logit there are only variables specific to the options being chosen.

Suppose a choice between different buses and trains to go from one district to another. A multinomial logit will use explanatory variables like being a male/female, having one/two/more kids, etc. The result of the multinomial logit will tell us if males are more likely to take this bus than females or if having kids makes it more likely to take the train rather than the bus.

A conditional logit will only have variables like the duration of the travel for each buses, the cost of the train/bus trip, etc... The result of the estimation will tell us what are the factors which are the more determinant in the choice between buses and trains: is it the time duration of the trip? its cost? the comfort?

Applications
Random multinomial logit models combine a random ensemble of multinomial logit models for use as a classifier.