How to analyze clustered or hierarchical data

01Why clustered data needs special treatment

Field surveys are seldom simple random samples of individuals. Pupils are sampled within schools, employees within companies, patients within clinics, and schools within regions. People in the same group resemble each other more than they resemble people elsewhere. Children in one school share teachers, resources, and neighbourhood. Employees in one company share managers, pay scales, and workplace culture.

This resemblance breaks a core assumption of ordinary regression, that each observation is independent. When it is ignored, the model behaves as though you have more independent information than you do. Standard errors come out too small, confidence intervals too narrow, and results look significant that are not. Multilevel regression, also called mixed-effects or hierarchical linear modelling, corrects this by modelling the group structure directly.

In the figure below, students in the same school share a colour, because they share teachers, resources, and surroundings. The same logic applies at every level of the structure.

The word hierarchical here describes nested data. It has no connection to hierarchical regression, which enters predictors in planned blocks.

02The intraclass correlation

The key number in a multilevel analysis is the intraclass correlation, or ICC. It states what share of the total variation in the outcome lies between groups rather than within them. An ICC of 0.15 for test scores across schools means 15% of the variation in scores is between schools, and 85% is between students within the same school.

A higher ICC means observations in the same group are more alike, and ignoring the clustering does more damage. In education research, ICCs of 0.10 to 0.25 are typical. In some health and community settings, values are lower. Even an ICC of 0.05 can matter when groups are large. An ICC near zero suggests grouping adds little, and an ordinary regression may do.

The ICC is also a finding in its own right. It tells you at what level most of the variation sits, and therefore whether changes aimed at schools, workplaces, or individuals have the most room to make a difference.

03Random intercepts and random slopes

A random intercept model lets each group have its own average level of the outcome. Some schools score higher on average and some lower, and the model estimates how much they vary. Predictors have the same effect in every group.

A random slope model goes further and lets the effect of a chosen predictor vary between groups. The benefit of an extra hour of study, for instance, might be larger in some schools than in others. This is a richer model but needs more groups and more data, and it should be used when theory suggests the effect really varies.

04When to use it

Use it when respondents are nested in groups and the group structure matters, or when the sampling design was clustered. It suits students in schools, patients in clinics, employees in companies, repeated measures on the same individuals over time, and multi-country surveys. It handles continuous, binary, and count outcomes.

If your data has no meaningful grouping, use linear, logistic, or Poisson regression. For a simple repeated-measures design without predictors, repeated-measures ANOVA may be enough.

How many groups do you need? As a guide, at least 20 to 30 groups are needed for stable estimates of the between-group variation, and more if you want random slopes. With very few groups, such as five regions, treat groups as a fixed predictor instead.

05The model and how it is calculated

The model

A multilevel model writes the outcome for student i in school j as a school-level baseline plus the effect of the predictors plus an individual error. The simplest version is the random intercept model.

The coefficients g00, b1 and g01 are the fixed effects, which apply to everyone. The term u(j) is the random effect, the amount by which school j sits above or below the average. It is assumed to follow a normal curve with variance σ²u, and the individual error e(ij) follows one with variance σ²e. A random slope model adds a school-specific term to the effect of a predictor, so that the effect itself varies between schools.

The parameters are estimated by maximum likelihood, usually in its restricted form, REML. The software chooses the fixed effects and the two variances that make the observed data most probable.

The intraclass correlation

In a model with no predictors, the example data give σ²u = 29.96 and σ²e = 153.01, so ICC = 29.96 ÷ (29.96 + 153.01) = 0.164. About 16% of the variation in scores lies between schools. The ICC shown in the results below comes from the model that includes the predictors, and it is 0.235. Study hours explain a lot of student-level variation, so what remains between schools is a larger share of what is left.

What ignoring clustering costs

Respondents in the same school carry less independent information than respondents in different schools. The design effect measures the loss:

With m = 25 students per school and the no-predictor ICC of 0.164, the design effect is 1 + 24 × 0.164 = 4.93. The 1,000 students carry about as much information as 203 independent ones. An ordinary regression would understate the standard errors of school-level predictors by a factor of about √4.93 = 2.22, which is enough to turn many non-significant results into apparently significant ones.

06Reading the output

Fixed effects

The main table shows the effect of each predictor on the outcome, averaged across groups. For continuous outcomes these are coefficients in the units of the outcome. For binary outcomes they can be shown as odds ratios, and for counts as incidence rate ratios. A toggle switches to raw coefficients. Their interpretation matches the single-level counterpart, with the added phrase that the effect is estimated after accounting for the grouping.

Variance components and ICC

This table gives the between-group variation and, for continuous outcomes, the within-group variation, together with the ICC and its confidence interval. With three or four levels, an ICC is reported for each level.

The test that grouping matters

For binary and count outcomes, the summary includes a likelihood-ratio test comparing the multilevel model with an ordinary model. A significant result supports the multilevel structure. If the test shows no improvement, the between-group variation is close to zero and a simpler model would do.

Overall test and fit

A Wald test checks whether the fixed effects together are significant. The summary also lists the log-likelihood, AIC, and BIC.

The ICC check

The module shows a second estimate of the ICC, computed directly from group averages with no modelling assumptions. If it agrees with the model-based value, you can trust the estimate. If the two diverge, treat the model-based ICC with caution. Binary and count models use an approximation that can understate group variation when there are few groups or when the ICC is high, and the module warns you when that may be the case.

07How to run it in AnalyZ Solutions

Open Inferential Analysis, choose Regression, then Multilevel Regression.

1. Choose the outcome type
   Continuous, binary, or count.
2. Choose the model type
   Random intercept, or random slope.
3. Choose the number of levels
   Two levels cover individuals within one type of group. Three and four levels add further layers, such as students within classes within schools.
4. Select the dependent variable
   The selector matches the outcome type. For a binary outcome, also choose which value is the event.
5. Select the grouping variables
   Choose one variable for each level above the individual level. A hierarchy check confirms that each lower group belongs to only one higher group, for example that each class sits inside one school.
6. Select the fixed-effect predictors
   Choose individual-level and group-level variables.
7. For a random slope, choose the variable whose effect may vary
   The effect of this predictor is allowed to differ across groups.
8. Click Run
   The output includes the model summary, the fixed-effects table, variance components with ICC, the estimated random effect for each group, and a plain-language interpretation.

08Interpreting the result

The figures below are invented to show the steps, and they follow the results tables in AnalyZ Solutions, with a few columns left out to save space. Suppose exam scores are recorded for 25 students in each of 40 schools. The predictors are weekly study hours (measured for each student) and whether the school provides free meals (a school-level variable), with a random intercept for school.

Reading it step by step

1. Start with the ICC
   The ICC is 0.235, so about 23% of the variation left after accounting for the predictors is between schools, and the rest is between students within the same school. This is enough clustering that an ordinary regression would have understated its standard errors.
2. Read the student-level effect
   Each extra weekly hour of study goes with 2.61 more points, with a narrow interval of 2.41 to 2.81. This effect is estimated within and across schools, after allowing for the school differences.
3. Read the school-level effect
   Schools that provide free meals score 5.20 points higher on average (interval 1.71 to 8.70, p 0.0036). The interval is much wider than the one for study hours, because this predictor only varies across the 40 schools, so there is far less information behind it.
4. Check the number of groups
   With 40 schools, the variance estimates are reasonably stable. With fewer than about 20 they would not be. A school-level result describes schools, and it should not be turned into a claim about individual students.

In plain words

Students in the same school resemble each other, with about 23% of the remaining variation in scores lying between schools. After accounting for this, more study time was strongly associated with higher scores, and schools that provide free meals scored about 5 points higher. The meals result is less precise than the study-hours result, and it is an association between schools.

09Common mistakes

Ignoring the clustering. This inflates significance and leads to false findings.

Too few groups. Fewer than about 20 groups give unreliable variance estimates.

Interpreting group-level predictors as individual effects. A school-level variable describes schools, so claims about individuals need care.

Adding a random slope by default. It uses up data. Add it only when there is a reason to expect the effect to vary.

Ignoring the ICC warning. If the model-based and direct ICC estimates disagree, say so in your report.