Counterfactual mediation splits an effect into the part that goes through one or more middle steps and the part that does not. It works with yes or no outcomes, covariates, serial mediators and groups, and it checks how fragile the result is. This guide covers the definitions, the method, every panel, a worked example and how to report it.
Counterfactual mediation answers the same question as mediation analysis: how much of an effect goes through a middle step? It does it in a way that works when your outcome or your middle step is a yes or no, or a set of ordered categories, and when you want to take other variables into account.
The word counterfactual means "what would have happened otherwise". The method compares the average result in situations that you can describe in words, such as "everyone trained" and "no one trained". The size of each effect is given in the units of the outcome. For a yes or no outcome, that is percentage points.
A programme offered job training to 900 young adults. Half took part, and taking part was voluntary. Six months later you know whether each person is employed (yes or no), and you have a confidence score and a job-search skills score. You also know age and sex. About 22% of the group are employed. You think training helps people into work by raising confidence and job-search skills. You want to know four things. How many percentage points does training add to the chance of being employed? How much of that goes through confidence, and how much through job-search skills? Is there any effect left that neither explains? And how sure can you be if some unmeasured factor is at work?
Use it when your outcome or mediator is yes or no or an ordered scale, when you want to adjust for other variables such as age and sex, when you have up to four mediators, or when you want the result in plain percentage points. It also covers an exposure with several categories, and results by group.
If your three variables are all numeric and you need nothing else, plain mediation analysis is quicker. For longer chains, use path analysis.
| Point | What is needed | If it is not met |
|---|---|---|
| Exposure | Numeric, yes or no, or a category variable with 3 to 10 categories. For categories, you pick a reference category. | Other types are not accepted. |
| Mediators | One to four. Numeric, yes or no, or ordered. Parallel if they do not affect each other, serial if they follow each other. | Wrong order in a serial set gives a wrong split. |
| Outcome | Numeric, yes or no, or an ordered scale. | Choose the type that matches the answers. |
| Covariates | Variables that could affect the mediator and outcome, such as age and sex. They are held fixed. | Missing important covariates leaves hidden common causes. |
| Sample size | Several hundred cases, and more for a yes or no outcome with few cases in one category. The example uses 900. | Intervals widen and some resamples fail. The module reports failed resamples. |
| Order | Exposure first, then mediators in order, then the outcome. Covariates are measured before the exposure. | A mediator measured after the outcome cannot mediate. |
Figure 1 shows the example. Training may raise confidence and job-search skills. Each of those may raise the chance of employment. Training may also help in ways the two mediators do not capture, which is the direct arrow. Age and sex are held constant in every step.
The method splits the total effect by comparing three situations, shown in Figure 2.
The step from the first to the second is the direct effect. It is what training does with the mediators held back. The step from the second to the third is the indirect effect. It is what changes only because the mediators change. The two together are the total effect.
In the example, the average chance of employment is about 15.2% with nobody trained, about 21.9% with training and mediators held back, and about 28.1% with training and mediators as they actually change. The direct effect is the first step, 21.9 - 15.2 = 6.7 points. The indirect effect is the second step, 28.1 - 21.9 = 6.2 points. The total is 12.9 points.
Imagine something you did not measure, such as motivation. It could make people more likely to join training and also more likely to find work. Then part of what looks like an effect of training is really motivation. The E-value says how strong that link would have to be to explain the result away.
For the total effect, the chance of employment goes from 15.2% to 28.1%, a risk ratio of 1.85. The E-value is 1.85 + √(1.85 × 0.85) = 3.09. An unmeasured factor would need to be linked to both training and employment by a risk ratio of about 3.1 each, beyond age and sex, to explain the total effect away. To reach an interval that includes zero, a factor of 2.38 would be enough.
Let X be the exposure (trained or not), M the mediator (confidence) and Y the outcome (employed). Write Y(x, m) for the outcome a person would have if X were set to x and M set to m. Write M(x) for the mediator they would have if X were set to x. We never see both versions for one person. The method asks what would happen on average if everyone were in each situation.
In words: the direct effect is the change in outcome if the exposure changes while the mediator stays where it would have been without the exposure. The indirect effect is the change if the exposure is held at its trained level and only the mediator moves from the level it would have without the exposure to the level it would have with it.
For a yes or no outcome, effects are differences in the share with the outcome. In the example 15.2% of untrained people would have had a job. The total effect is 12.9 percentage points, so 28.1% of everyone would have a job if all were trained. Of the 12.9, 6.7 points are direct and 6.2 points go through confidence, so about 48% is indirect.
Parallel mediators each carry part of the effect and do not affect each other. The module gives the effect through each, and tests whether two routes differ. Serial mediators follow an order, such as training, then knowledge, then confidence. The order matters, because the model assumes each mediator can affect the ones after it. The module splits the indirect effect by route. Put the mediators in the order you believe in and say why. With the exposure-mediator interaction switched on, the effect of the mediator may differ for trained and untrained people.
With a categorical exposure, each category is compared with the reference category, and the module gives total, direct and indirect effects for each comparison. With results by group (two to six groups), the whole analysis runs inside each group. The groups are compared using paired bootstrap resamples, so the difference in effects has its own interval.
The result needs no hidden factor that affects both the mediator and the outcome. The sensitivity panel shows how fragile the result is to that assumption in two ways.
The E-value is the minimum strength of link, on the risk ratio scale, that a hidden factor would need with both the exposure and the outcome to fully explain the effect away. In the example, the E-value is 3.09 for the total effect and 1.89 for the indirect effect (2.38 and 1.57 for the interval limits closest to no effect). A hidden factor would need to be linked to both training and employment by a risk ratio of about 3 to explain away the total effect. For the indirect effect, a risk ratio of about 1.9 would do it, so the indirect result is less sturdy than the total.
The hidden-correlation curve shows the indirect effect as a function of the correlation between the errors of the mediator and outcome models. The point where the curve crosses zero tells how much hidden correlation would erase the indirect effect.
The table lists the total, direct and indirect effects, with the indirect effect split by mediator. Each has an interval from the resampling. The chart shows the same as a forest plot. An interval that does not include zero is a clear effect.
With several mediators, the module tests whether two routes differ in size. If the interval of the difference includes zero, you cannot say one mediator matters more than the other.
This is the indirect effect divided by the total effect, with an interval. Treat it as a rough guide, and be careful when the total effect is small.
The mediator and outcome models are shown, so you can check the direction of every link.
A collapsed panel gives E-values for the total, direct and indirect effects. When there is one continuous mediator and a continuous outcome, it also shows how the indirect effect changes if the unmeasured factors behind the mediator and the outcome are correlated.
If the exposure has three to ten categories, the module compares each category with a reference category you choose. It can also run the model separately for each group, such as women and men, and test whether the effects differ between groups.
Open Structural Equation Modelling and choose Counterfactual Mediation. The module is marked Beta.
| Panel | What it shows | What to look for |
|---|---|---|
| Effects table and chart | Total, direct and indirect effects, with intervals, in percentage points or in the outcome's units. | An interval that avoids zero. |
| Effects by route | The indirect effect through each mediator, and tests of differences. | Which route carries most. |
| Share mediated | Indirect divided by total, with a note when it is not meaningful. | Shown only when the effects point the same way. |
| The models | The coefficients of the mediator and outcome models. | Check that signs are sensible. |
| Sensitivity | E-values and the hidden-correlation curve. | Values close to 1 mean the result is fragile. |
| By group | Effects for each group and the differences. | Whether the difference interval avoids zero. |
| Interpretation | A plain-language summary. | A first draft. |
The figures below are invented to show the steps. They come from a real run of the module on a simulated dataset of 900 young adults, with 1,000 resamples. The exposure is training (yes or no), the mediators are confidence and job-search skills, and age and sex are covariates. The outcome is employed (yes or no).
| Effect | Estimate | 95% CI | p-value |
|---|---|---|---|
| Total effect | 12.9 | 7.7 to 18.2 | 0.001 |
| Direct effect | 6.7 | 1.0 to 12.1 | 0.020 |
| Indirect effect | 6.2 | 3.3 to 9.0 | 0.001 |
| through confidence | 3.8 | 1.8 to 5.9 | 0.001 |
| through job-search skills | 2.4 | 0.7 to 4.5 | 0.002 |
Step 1: read the total effect. Training adds about 12.9 percentage points to the chance of being employed (interval 7.7 to 18.2). That is from 15.2% without training to about 28.1% with it.
Step 2: split it. About 6.2 points are indirect and 6.7 points are direct. Both intervals stay above zero. Confidence carries 3.8 points and job-search skills 2.4.
Step 3: compare the routes. The difference between the two routes is 1.3 points (interval -1.4 to 4.1). That includes zero, so the data do not show one mediator to be more important than the other.
Step 4: read the share mediated. About 48% of the total effect goes through the two mediators (interval 25% to 86%). The interval is wide, so the share is only a rough guide.
| Effect | Estimate (points) | E-value | E-value to reach an interval that includes zero |
|---|---|---|---|
| Total effect | 12.9 | 3.09 | 2.38 |
| Indirect effect | 6.2 | 1.89 | 1.57 |
Step 5: check the sensitivity. To explain the total effect away, a hidden factor would need to be linked to both training and employment by a risk ratio of about 3.1 each. For the indirect effect, the figure is 1.9. That is a weaker link, so the indirect result is easier to challenge than the total. Whether such a factor is likely depends on what you know about how people joined the programme.
Young adults who took the job training were about 13 percentage points more likely to be employed six months later. About half of that gain went through higher confidence and better job-search skills. The rest was not explained by these two. The data cannot say which of the two mediators matters more. These results would need a fairly strong hidden factor to disappear, but the indirect part is less secure than the total. As the training was not assigned at random, differences in motivation could still play a part.
Reading the result as proof of cause. It is an estimate of a causal effect under assumptions. Use the sensitivity panel and say so in your report.
Adjusting for the wrong things. Include variables that come before the exposure. Do not adjust for variables that the exposure changes, because that removes part of the effect you want to measure.
Ignoring the order of mediators. In a serial model with a yes or no variable, the split of routes can depend on the order, so choose the order from your theory.
Too many mediators for the sample. Each adds a model. Keep to what your theory needs.
Treating a wide interval as no effect. It may mean too little data to tell.
Comparing groups without checking group sizes. Small groups give wide, unstable intervals.
Open Structural Equation Modelling in AnalyZ Solutions. Free to sign up, free to analyze.
Get started