How to tell if two factors, and their combination, affect an outcome

01What is two-way ANOVA?

One-way ANOVA compares group averages across one grouping variable. Many real questions involve two. Two-way ANOVA studies two grouping variables at the same time. Each grouping variable is called a factor, and each category within it is a level. Crossing the two factors creates cells, one for every combination of levels. Gender with two levels and delivery method with two levels, for example, gives four cells: women online, women in person, men online, and men in person.

Like one-way ANOVA, it works by splitting the variation in the outcome. Here the variation is divided into four parts: variation explained by the first factor, by the second factor, by the interaction between them, and variation left unexplained within the cells. Each effect is compared with the unexplained variation to give an F statistic and a significance level.

Three questions at once

Two-way ANOVA answers three questions in one analysis.

• Does the first factor matter? This is the main effect of gender, the difference between women and men averaged over both delivery methods.
• Does the second factor matter? This is the main effect of delivery method, the difference between online and in person averaged over both genders.
• Does the effect of one factor depend on the other? This is the interaction.

The third question is often the most interesting. An interaction means the combination matters, and the factors cannot be described one at a time. If online delivery raises scores for women and lowers them for men, the overall online average may look the same as the in-person average, because the opposite effects cancel out. The main effects would suggest that delivery does not matter, when it matters a great deal for each gender.

Seeing an interaction

The easiest way to spot an interaction is to plot the cell averages, with one line for each level of one factor. Parallel lines mean the effect of the factor on the horizontal axis is the same for both groups, so there is no interaction. Lines that cross, or that fan out so that the gap changes, mean the effect differs between groups.

Why not run two one-way ANOVAs?

Two separate analyses would each ignore the other factor. That wastes information, because variation due to the second factor is left in the unexplained part and makes each test less sensitive. It also makes it impossible to detect an interaction, since each analysis sees only one factor at a time. Analysing both together is more informative and uses the data more efficiently.

A balanced design is easiest

Two-way ANOVA is easiest to read when every cell holds about the same number of cases, which is called a balanced design. Experiments tend to produce balanced designs, while surveys rarely do. When cell sizes differ a lot, the effects of the two factors overlap, and the results depend on how the software divides up the shared variation.

02Two-way ANOVA compared with one-way ANOVA and regression

Two-way ANOVA is a regression with two categorical predictors and their interaction. In a balanced design, the F tests for the two main effects and the interaction match the ones from a regression fitted to the same data, so the choice does not change the conclusion. The general relationship between ANOVA and regression is covered in the one-way ANOVA guide.

The practical difference is flexibility. Regression can add numeric predictors such as age, can include more than two factors, and copes better when cell sizes are unequal. In AnalyZ Solutions, however, the regression module has no interaction terms. Two-way ANOVA is therefore the way to test whether the effect of one categorical factor depends on another.

03When to use two-way ANOVA

Use it when all of these hold:

• The outcome is numeric, such as a test score, a measurement, a spending amount, or a scale total.
• You have two categorical factors that cross each other, so every combination of levels has respondents.
• Each person or case falls into exactly one cell, so the observations are independent.
• You care about both factors and about whether their effects depend on each other.

Typical examples are test scores by gender and course delivery, crop yield by fertiliser and irrigation method, customer spending by store type and season, and recovery time by treatment and age group.

Choose a different method when your situation changes:

• For one grouping variable, use one-way ANOVA.
• For the same people measured several times, use repeated-measures ANOVA.
• For numeric covariates, more than two factors, or very unequal cell sizes, use linear regression, keeping in mind that the module has no interaction terms.
• For a yes or no outcome, use logistic regression.
• For respondents nested in clusters such as schools or clinics, use multilevel regression.
• For a skewed or ordinal outcome, AnalyZ Solutions has no rank-based two-way test. Consider transforming the outcome, or analysing each factor separately with Kruskal-Wallis.

04Assumptions

The assumptions match one-way ANOVA: independent observations, roughly normal outcomes within each cell, and similar spread across cells. Two-way ANOVA is also easiest to interpret when the cells are of roughly similar size. When some cells are much smaller than others, the main effect estimates become harder to separate from each other, and results can differ between software packages that partition the variation in different ways. Aim for at least 10 to 15 respondents per cell if you can.

05Reading the result

Start with the interaction

If the interaction is significant, interpret it first. The effect of each factor depends on the other, so statements about main effects alone can mislead. The interpretation in AnalyZ Solutions leads with the interaction when it is significant, telling you that the effect of one factor depends on the level of the other.

To understand the pattern, read the cell averages in the descriptives table. Compare, for instance, the online and in-person averages separately for women and for men.

Then the main effects

If the interaction is not significant, the main effects can be read on their own. Each one asks whether the outcome differs across the levels of that factor when the other factor is averaged over.

Eta-squared

Each row of the table has an eta-squared, the share of total variation explained by that effect. Under 6% is small, 6% to 14% medium, and over 14% large.

06How to run it in AnalyZ Solutions

Open Inferential Analysis, choose Compare Means, then Two-way ANOVA.

1. Select the dependent variable
   Choose the numeric outcome.
2. Select Factor A
   Choose the first categorical grouping variable.
3. Select Factor B
   Choose the second one. It must be a different variable.
4. Apply a subsample filter if needed
   For instance, to analyse one region only.
5. Click Run
   The output includes a table with the average, standard deviation, and count for every combination of the two factors, an ANOVA table with rows for each factor, the interaction, and the error, and a plain-language interpretation.

The table does not include automatic follow-up comparisons between cells. If the interaction is significant, use the cell averages to describe the pattern, and run a t-test within one level of a factor if you need a specific comparison.

07Interpreting the result

The figures below are invented to show the steps, and they follow the results tables in AnalyZ Solutions, with a few columns left out to save space. Suppose test scores were recorded for 20 students in each combination of gender and course delivery (online or in person).

Reading it step by step

1. Look at the interaction first
   The Gender × Delivery row has a p-value below 0.001 with three stars, so the effect of delivery depends on gender. This finding takes priority over the other two rows.
2. Explain it with the cell averages
   Women scored 78 online and 70 in person, while men scored 68 online and 76 in person. Online delivery helped women by 8 points and cost men 8 points. The pattern reverses between the two genders.
3. Read the main effects with that in mind
   Gender and Delivery are both non-significant (p 0.2671 and 1.0000). That does not mean neither matters. The two opposite effects cancel out when averaged over the other factor, which is why the interaction has to be read first.
4. Judge the size
   The interaction has an eta-squared of 0.206, so it explains about 21% of the variation in scores, a large effect. The main effects explain almost nothing on their own.

In plain words

The best delivery method depends on gender. Women did better online, and men did better in person, and the interaction is significant at the 0.1% level and large. Reporting either factor alone would have suggested that neither matters. Because the module gives no follow-up tests between cells, describe the pattern from the cell averages, and run a separate t-test within one gender if you need a formal comparison.

08Common mistakes

Interpreting main effects when an interaction is present. The main effects average over a pattern that differs between levels, and the average can describe nobody.

Empty or tiny cells. If one combination has very few respondents, its average is unreliable. Check the counts in the descriptives table.

Treating a non-significant interaction as proof of parallel effects. Interaction tests have low power, especially in small samples.

Using it for more than two factors. Add further predictors through linear regression.