What is multinomial regression? Theory, calculation and application

01What is multinomial regression?

Multinomial regression is a method for studying outcomes that fall into three or more categories with no natural order. It shows which factors are linked to each category, how strongly, and how likely each category is for a particular person.

Many outcomes are like this: the main way people travel to work (car, bus, bicycle, on foot), the type of housing a family lives in (owned, rented, shared), or which of three course options a student chooses. The thing you want to explain is the outcome, and the factors you use to explain it are the predictors. Each possible result of the outcome is a category.

An example

Suppose you ask 600 people how they usually travel to work: by bus, by car, or by bicycle. You also know whether each person has a high or a low income and whether they live in the city centre or outside it. You want to know three things. Does income change how people travel? Does living in the centre? And what is the chance of each choice for a particular person, such as someone on a high income who lives in the centre? Multinomial regression answers all three.

Why not use a different method?

With two categories, logistic regression is the standard choice. With three or more, you could run one logistic regression per category against all the others, but each would ignore the rest and their predicted chances would not add up to 100%. A chi-square test shows whether the pattern differs between groups but cannot adjust for several factors at once. Multinomial regression models all the categories together, so the predicted chances always add up to 100%.

What you get out of it

1. Direction
   Whether each predictor makes a category more or less likely.
2. Size
   How much, usually as a relative risk ratio, which a later section explains.
3. Confidence
   Whether the pattern could easily be chance, shown by a significance level and a confidence interval.
4. Prediction
   The chance of every category for any combination of predictor values.

02When to use it

Use multinomial regression when the outcome has three or more categories without a natural order. If the categories are ordered, such as low, medium, and high, use ordinal regression, which is more economical and easier to interpret. For two categories, use logistic regression.

The method needs adequate sample size. Each outcome category needs enough cases, and models with many predictors and small categories often fail to converge or give unstable results.

03The building blocks: chances, a reference category and comparisons

Multinomial regression rests on three ideas: chances that add up to 100%, a reference category, and a set of comparisons against that reference.

Chances that add up to 100%

Every person has a chance of choosing each category, and together those chances add up to 100%. A person might have a 45% chance of taking the bus, a 38% chance of driving and a 17% chance of cycling. Raising one chance necessarily lowers the others, so the categories have to be modelled together.

The reference category

The model picks one category as the reference, such as the bus, and describes every other category by comparing it with the reference. With three categories there are two comparisons: car against bus, and bicycle against bus. Each comparison has its own coefficients, so a predictor can have different effects in different comparisons. High income might make driving much more likely than taking the bus while making no difference to cycling. The choice of reference changes the numbers but not the conclusions.

How a comparison is expressed

Odds compare how likely one thing is with how likely another is. If the car has a 40% chance and the bus a 20% chance, the odds of car against bus are 2 to 1. Taking the natural logarithm of the odds gives the log-odds, which a straight line can model without producing impossible values, exactly as in logistic regression. The model fits one straight line for each comparison, then converts the results back into chances that add up to 100%.

The figure shows what the model produces for two people.

The next section writes the model in symbols and shows how the chances are calculated.

04The model and how it is calculated

The model

With a reference category, say Bus, the model fits one equation for each other category. Each equation gives the log of the chance of that category relative to the reference.

Raising e to a coefficient gives its relative risk ratio, which Reading the output explains in plain terms. To turn the equations into probabilities, call each right-hand side an index, with the reference category's index fixed at zero:

The probabilities always add up to 1. Coefficients are found by maximum likelihood, as in logistic regression.

Making a prediction

Take someone with a high income who lives in the city centre.

1. Indices
   From the results below, the Car index is -0.259 and the Bicycle index is 0.112.
2. Exponentiate
   e^-0.259 = 0.772 and e^0.112 = 1.119.
3. Add 1 and divide
   The denominator is 1 + 0.772 + 1.119 = 2.891.
4. Probabilities
   Bus = 1 ÷ 2.891 = 0.346, Car = 0.772 ÷ 2.891 = 0.267, and Bicycle = 0.387. They add up to 1.

For someone with a low income who lives outside the centre, the same steps give 45% for the bus, 38% for the car, and 17% for the bicycle. The shift between the two profiles combines both coefficients across all three categories, which a single ratio cannot show.

05Reading the output

Relative risk ratios

Multinomial models report each effect as a relative risk ratio, often shortened to RRR. It compares the chance of an outcome category with the chance of the baseline category, for a one-unit rise in a predictor or for one group compared with another.

A ratio of 1 means no difference. Above 1, the outcome becomes more likely relative to the baseline, and below 1, less likely. If the baseline is taking the bus and the ratio for driving is 2.5 for people with a high income compared with a low income, then a high income makes driving 2.5 times more likely than taking the bus, relative to the low-income group.

Note the phrase relative to the baseline. A ratio compares two outcome categories at a time. It says nothing on its own about how the chance of the category changes in absolute terms, and the choice of baseline changes every number in the output. Pick the baseline that gives your readers the most natural comparison, usually the largest category or the status quo.

Coefficients by outcome category

The coefficient table is organised by outcome category. Each block compares one category with the baseline and lists the effect of every predictor. Each row gives the relative risk ratio with a 95% confidence interval and significance level. A toggle switches the table to raw coefficients, which are on the log scale.

The plot

The module draws a panel for each outcome category, so you can see at a glance which predictors matter for which comparison.

Model fit

The summary reports the number of cases, the number of outcome categories, the reference category, the log-likelihood, a likelihood-ratio test, a pseudo R-squared, and AIC.

The interpretation

The plain-language text describes significant effects in terms of being more or less likely to fall in one category than the baseline, and it flags estimates that are unstable.

06How to run it in AnalyZ Solutions

Open Inferential Analysis, choose Regression, then Multinomial Regression.

1. Select the dependent variable
   Choose an outcome with three or more categories.
2. Choose the reference category
   Pick the outcome category everything else will be compared with.
3. Select the independent variables
   Choose numeric or categorical predictors.
4. Set reference categories for categorical predictors
   Categorical predictors are converted to comparisons against a reference category automatically. Pick the reference category you want each group compared with, usually the largest or the most natural comparison group.
5. Apply a subsample filter if needed.
6. Click Run regression
   You receive the coefficient tables by outcome category, the model summary, the panels of coefficient plots, and the interpretation.

07Interpreting the result

The figures below are invented to show the steps, and they follow the results tables in AnalyZ Solutions, with a few columns left out to save space. Suppose 600 people report how they usually travel to work: by bus (the reference category), by car, or by bicycle. The predictors are high income (reference: Low income) and living in the city centre (reference: Outside).

Reading it step by step

1. Name the comparison
   Every ratio compares one outcome category with the reference category, Bus. The first table compares Car with Bus, and the second compares Bicycle with Bus. Change the reference and every number changes with it.
2. Read the car results
   High income has a relative risk ratio of 2.750, with an interval of 1.843 to 4.104. Compared with people on low incomes, high earners are about 2.75 times more likely to drive than to take the bus. Living in the city centre has a ratio of 0.335, so central residents are about one third as likely to drive rather than take the bus.
3. Read the bicycle results
   City centre residents are about 2.56 times more likely to cycle than to take the bus (interval 1.663 to 3.942). High income has a ratio of 1.153 with a p-value of 0.503, and its interval, 0.760 to 1.747, includes 1, so there is no evidence that income changes cycling relative to the bus.
4. Remember what a ratio does not say
   A relative risk ratio compares two categories. It does not say how the overall share of drivers changes. Model fit is modest, with a pseudo R² of 0.080, and the likelihood-ratio test is significant at the 0.1% level.

In plain words

Income and location both shape how people get to work. Higher earners are much more likely to drive than to take the bus, and city centre residents are less likely to drive and more likely to cycle. Income makes no clear difference to cycling versus the bus. The results describe associations and depend on the choice of the bus as the reference.

08Common mistakes

Forgetting the baseline. Every ratio compares an outcome category with the baseline. State it every time.

Reading a ratio as a change in probability. It compares two outcome categories with each other.

Using too many predictors for small categories. Each outcome comparison behaves like a separate model and needs adequate cases.

Treating ordered outcomes as unordered. This discards information. Use ordinal regression.