6. The Multinomial Logit Regression Model


Suppose we have a nominal dependent variable such as the mode of transportation (walk, bike, bus, and car). The multinomial logit and conditional logit models are commonly used; the multinomial probit model is not often used mainly due to the practical difficulty in estimation. However, Stata does have the .mprobit command to fit the model.

In the multinomial logit model, the independent variables contain characteristics of individuals, while they are the attributes of the choices in the conditional logit model. In other words, the conditional logit estimates how alternative-specific, not individual-specific, variables affect the likelihood of observing a given outcome (Long 2003). Therefore, data need to be appropriately arranged in advance.

6.1 Multinomial Logit/Probit in Stata (.mlogit and .mprobit)

Stata has the .mlogit command for the multinomial logit model. The base() option indicates the value of the dependent variable to be used as the base category for the estimation. You may omit the default option, base(0).

. mlogit transmode income age male, base(0)

Iteration 0:   log likelihood = -444.84113
Iteration 1:   log likelihood = -411.18604
Iteration 2:   log likelihood = -406.36474
Iteration 3:   log likelihood =  -406.3251
Iteration 4:   log likelihood = -406.32509
 
Multinomial logistic regression                   Number of obs   =        437
                                                  LR chi2(9)      =      77.03
                                                  Prob > chi2     =     0.0000
Log likelihood = -406.32509                       Pseudo R2       =     0.0866
 
------------------------------------------------------------------------------
   transmode |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1            |
      income |   4.018021    1.34443     2.99   0.003     1.382986    6.653057
         age |   .1915917   .1392928     1.38   0.169    -.0814172    .4646006
        male |   .2582886   .4039971     0.64   0.523    -.5335311    1.050108
       _cons |  -6.903473    2.97678    -2.32   0.020    -12.73785   -1.069091
-------------+----------------------------------------------------------------
2            |
      income |   8.951041   1.338539     6.69   0.000     6.327552    11.57453
         age |   .1374997   .1451938     0.95   0.344    -.1470749    .4220742
        male |   .1573179   .4191014     0.38   0.707    -.6641057    .9787415
       _cons |  -9.091051   3.088123    -2.94   0.003    -15.14366   -3.038442
-------------+----------------------------------------------------------------
3            |
      income |   4.210485   1.032024     4.08   0.000     2.187755    6.233215
         age |   .3457236   .0995071     3.47   0.001     .1506932     .540754
        male |   .5402549   .2769887     1.95   0.051    -.0026329    1.083143
       _cons |  -8.388756   2.135792    -3.93   0.000    -12.57483   -4.202681
------------------------------------------------------------------------------
(transmode==0 is the base outcome)

Let us see if the base outcome changes. As shown in the following, the parameter estimates and standard errors are changed, whereas the goodness-of-fit remains unchanged. The two .mlogit commands with different bases fit the same model but present the result in different manner. The SAS CATMOD procedure in the next section uses the largest value as the base outcome.

. mlogit transmode income age male, base(3)

Iteration 0:   log likelihood = -444.84113
Iteration 1:   log likelihood = -411.18604
Iteration 2:   log likelihood = -406.36474
Iteration 3:   log likelihood =  -406.3251
Iteration 4:   log likelihood = -406.32509
 
Multinomial logistic regression                   Number of obs   =        437
                                                  LR chi2(9)      =      77.03
                                                  Prob > chi2     =     0.0000
Log likelihood = -406.32509                       Pseudo R2       =     0.0866
 
------------------------------------------------------------------------------
   transmode |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
0            |
      income |  -4.210485   1.032024    -4.08   0.000    -6.233215   -2.187755
         age |  -.3457236   .0995071    -3.47   0.001     -.540754   -.1506932
        male |  -.5402549   .2769887    -1.95   0.051    -1.083143    .0026329
       _cons |   8.388756   2.135792     3.93   0.000     4.202681    12.57483
-------------+----------------------------------------------------------------
1            |
      income |  -.1924639    1.00606    -0.19   0.848    -2.164305    1.779377
         age |   -.154132   .1131506    -1.36   0.173    -.3759031    .0676392
        male |  -.2819663   .3443963    -0.82   0.413    -.9569706    .3930379
       _cons |   1.485283   2.430912     0.61   0.541    -3.279216    6.249783
-------------+----------------------------------------------------------------
2            |
      income |   4.740556   .9447126     5.02   0.000     2.888953    6.592158
         age |  -.2082239   .1164954    -1.79   0.074    -.4365507    .0201028
        male |   -.382937   .3490247    -1.10   0.273    -1.067013    .3011389
       _cons |  -.7022953   2.460119    -0.29   0.775     -5.52404    4.119449
------------------------------------------------------------------------------
(transmode==3 is the base outcome)

The SPost .mlogtest command conducts a variety of statistical tests for the multinomial logit model. This command supports not only Wald and likelihood ratio tests, but also Hausman and Small-Hsiao tests for the independence of irrelevant alternatives (IIA) assumption. The .mlogtest command works with the .mlogit command only.

. mlogtest, hausman smhsiao base

**** Hausman tests of IIA assumption
 
 Ho: Odds(Outcome-J vs Outcome-K) are independent of other alternatives.
 
 Omitted |      chi2   df   P>chi2   evidence
---------+------------------------------------
       0 |     0.260    8    1.000   for Ho   
       1 |    -3.307    8    1.000   for Ho   
       2 |    -0.319    8    1.000   for Ho   
       3 |     2.315    8    0.970   for Ho   
----------------------------------------------
 
**** Small-Hsiao tests of IIA assumption
 
 Ho: Odds(Outcome-J vs Outcome-K) are independent of other alternatives.
 
 Omitted |  lnL(full)  lnL(omit)    chi2   df   P>chi2   evidence
---------+---------------------------------------------------------
       0 |   -120.685   -116.139   9.092    4    0.059   for Ho   
       1 |   -131.938   -128.574   6.728    4    0.151   for Ho   
       2 |   -155.078   -150.308   9.540    4    0.049   against Ho
       3 |    -71.735    -67.571   8.327    4    0.080   for Ho   
-------------------------------------------------------------------

The Stata .mprobit command fits the multinomial probit model. The model took longer time to converge than the multinomial logit model.

. mprobit transmode income age male

Iteration 0:   log likelihood =  -425.2053 
Iteration 1:   log likelihood = -407.95972 
Iteration 2:   log likelihood = -406.38652 
Iteration 3:   log likelihood = -406.38431 
Iteration 4:   log likelihood = -406.38431 
 
Multinomial probit regression                     Number of obs   =        437
                                                  Wald chi2(9)    =      64.47
Log likelihood = -406.38431                       Prob > chi2     =     0.0000
 
------------------------------------------------------------------------------
   transmode |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
_outcome_2   |
      income |   2.651949   .8328566     3.18   0.001      1.01958    4.284318
         age |   .1501467   .0903614     1.66   0.097    -.0269584    .3272519
        male |   .1967795    .262047     0.75   0.453    -.3168232    .7103822
       _cons |  -5.075328   1.953866    -2.60   0.009    -8.904835   -1.245822
-------------+----------------------------------------------------------------
_outcome_3   |
      income |   5.757611   .8443105     6.82   0.000     4.102793    7.412429
         age |   .1218625   .0942662     1.29   0.196    -.0628959    .3066209
        male |   .1662947   .2772189     0.60   0.549    -.3770444    .7096339
       _cons |  -6.547874   2.031953    -3.22   0.001    -10.53043   -2.565319
-------------+----------------------------------------------------------------
_outcome_4   |
      income |   2.751622   .6936632     3.97   0.000     1.392067    4.111177
         age |   .2760071    .074178     3.72   0.000     .1306208    .4213933
        male |   .4232271   .2086763     2.03   0.043     .0142289    .8322252
       _cons |  -6.375609   1.598767    -3.99   0.000    -9.509134   -3.242083
------------------------------------------------------------------------------

Top

6.2 Multinomial Logit in SAS

SAS has the CATMOD procedure for the multinomial logit model. In the CATMOD procedure, the RESPONSE statement is used to specify the functions of response probabilities.

PROC CATMOD DATA = masil.students;
   DIRECT income age male;
   RESPONSE LOGITS;
   MODEL transmode = income age male /NOPROFILE;
RUN;

                                      The CATMOD Procedure
 
                                         Data Summary
 
                     Response           transmode     Response Levels    4
                     Weight Variable    None          Populations      414
                     Data Set           STUDENTS      Total Frequency  437
                     Frequency Missing  0             Observations     437
 
 
                                   Maximum Likelihood Analysis
 
                           Maximum likelihood computations converged.
 
 
                            Maximum Likelihood Analysis of Variance
 
                       Source               DF   Chi-Square    Pr > ChiSq
                       --------------------------------------------------
                       Intercept             3        15.91        0.0012
                       income                3        45.72        <.0001
                       age                   3        14.66        0.0021
                       male                  3         4.73        0.1927
 
                       Likelihood Ratio    1E3       778.33        1.0000
 
 
                           Analysis of Maximum Likelihood Estimates
 
                        Function               Standard        Chi-
              Parameter  Number     Estimate      Error      Square    Pr > ChiSq
              -------------------------------------------------------------------
              Intercept    1          8.3888     2.1358       15.43        <.0001
                           2          1.4853     2.4309        0.37        0.5412
                           3         -0.7023     2.4601        0.08        0.7753
              income       1         -4.2105     1.0320       16.65        <.0001
                           2         -0.1925     1.0061        0.04        0.8483
                           3          4.7406     0.9447       25.18        <.0001
              age          1         -0.3457     0.0995       12.07        0.0005
                           2         -0.1541     0.1132        1.86        0.1731
                           3         -0.2082     0.1165        3.19        0.0739
              male         1         -0.5403     0.2770        3.80        0.0511
                           2         -0.2820     0.3444        0.67        0.4129
                           3         -0.3829     0.3490        1.20        0.2726

As mentioned before, the CATMOD procedure uses the largest value of the dependent variable as a base outcome. Accordingly, you need to compare the above with the Stata output of the base(3) option. The two outputs are the same except for the likelihood ratio.

Top

6.3 Multinomial Logit in LIMDEP (Mlogit$)

In LIMDEP, you may use either the Mlogit$ or simply the Logit$ commands to fit the multinomial logit model. Both commands produce the identical result. Like Stata, LIMDEP by default uses the smallest value as the base outcome.

MLOGIT;
   Lhs=transmod;
   Rhs=ONE,income,age,male$

Or,

LOGIT;
   Lhs=transmod;
   Rhs=ONE,income,age,male$

Normal exit from iterations. Exit status=0.
 
+---------------------------------------------+
| Multinomial Logit Model                     |
| Maximum Likelihood Estimates                |
| Model estimated: Sep 19, 2005 at 09:19:23AM.|
| Dependent variable             TRANSMOD     |
| Weighting variable                 None     |
| Number of observations              437     |
| Iterations completed                  6     |
| Log likelihood function       -406.3251     |
| Restricted log likelihood     -444.8411     |
| Chi squared                    77.03209     |
| Degrees of freedom                    9     |
| Prob[ChiSqd > value] =         .0000000     |
+---------------------------------------------+
+---------+--------------+----------------+--------+---------+----------+
|Variable | Coefficient  | Standard Error |b/St.Er.|P[|Z|>z] | Mean of X|
+---------+--------------+----------------+--------+---------+----------+
          Characteristics in numerator of Prob[Y = 1]
 Constant    -6.903472671       2.9767801   -2.319   .0204
 INCOME       4.018021309       1.3444306    2.989   .0028     .61683982
 AGE          .1915916653       .13929281    1.375   .1690     20.691076
 MALE         .2582886230       .40399708     .639   .5226     .57208238
          Characteristics in numerator of Prob[Y = 2]
 Constant    -9.091051495       3.0881231   -2.944   .0032
 INCOME       8.951040805       1.3385396    6.687   .0000     .61683982
 AGE          .1374996725       .14519378     .947   .3436     20.691076
 MALE         .1573178944       .41910143     .375   .7074     .57208238
          Characteristics in numerator of Prob[Y = 3]
 Constant    -8.388756169       2.1357918   -3.928   .0001
 INCOME       4.210485161       1.0320242    4.080   .0000     .61683982
 AGE          .3457236198   .99507140E-01    3.474   .0005     20.691076
 MALE         .5402549359       .27698869    1.950   .0511     .57208238
 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.)
 
+--------------------------------------------------------------------+
| Information Statistics for Discrete Choice Model.                  |
|                            M=Model MC=Constants Only   M0=No Model |
| Criterion F (log L)     -406.32509        -444.84113    -605.81064 |
| LR Statistic vs. MC       77.03209            .00000        .00000 |
| Degrees of Freedom         9.00000            .00000        .00000 |
| Prob. Value for LR          .00000            .00000        .00000 |
| Entropy for probs.       406.32509         444.84113     605.81064 |
| Normalized Entropy          .67071            .73429       1.00000 |
| Entropy Ratio Stat.      398.97109         321.93900        .00000 |
| Bayes Info Criterion     867.36958         944.40167    1266.34067 |
| BIC - BIC(no model)      398.97109         321.93900        .00000 |
| Pseudo R-squared            .08658            .00000        .00000 |
| Pct. Correct Prec.        64.98856            .00000      25.00000 |
| Means:       y=0    y=1    y=2    y=3    yu=4   y=5,    y=6   y>=7 |
| Outcome     .1648  .0892  .0961  .6499  .0000  .0000  .0000  .0000 |
| Pred.Pr     .1648  .0892  .0961  .6499  .0000  .0000  .0000  .0000 |
| Notes: Entropy computed as Sum(i)Sum(j)Pfit(i,j)*logPfit(i,j).     |
|        Normalized entropy is computed against M0.                  |
|        Entropy ratio statistic is computed against M0.             |
|        BIC = 2*criterion - log(N)*degrees of freedom.              |
|        If the model has only constants or if it has no constants,  |
|        the statistics reported here are not useable.               |
+--------------------------------------------------------------------+
Frequencies of actual & predicted outcomes
Predicted outcome has maximum probability.
 
            Predicted
------  --------------------  +  -----
Actual      0    1    2    3  |  Total
------  --------------------  +  -----
  0         5    0    0   67  |     72
  1         0    0    1   38  |     39
  2         0    0    1   41  |     42
  3         6    0    0  278  |    284
------  --------------------  +  -----
Total      11    0    2  424  |    437

Note that the variable name TRANSMOD was truncated because LIMDEP allows up to eight characters for a variable name. LIMDEP and Stata produce the same result of the multinomial logit model.

6.4 Multinomial Logit in SPSS

SPSS has the Nomreg command to estimate the multinomial logit model. Like SAS, SPSS by default uses the largest value as the base outcome; you may change the baseline by specifying FIRST or any particular value at the Base= option. When using point-and-click method, you need put categorical variables in a box labeled as "Factor(s)" and continuous variables in the "Covariate(s)" box.

NOMREG transmode BY male WITH income age
   /CRITERIA CIN(95) DELTA(0) MXITER(100) MXSTEP(5) CHKSEP(20) LCONVERGE(0)
   PCONVERGE(0.000001) SINGULAR(0.00000001)
   /MODEL /INTERCEPT INCLUDE /PRINT PARAMETER SUMMARY LRT .



Up: Table of Contents
Next: The Conditional Logit Model
Prev: Ordered Logit/Probit Models