How to Calculate Sample Size to Detect a Change in a Numeric Outcome

Many programme evaluations depend on comparing a numeric average across two groups, or across time. Did average income rise? Did test scores improve? Did the number of clinic visits per household increase? This tool answers that question when the outcome is a continuous number rather than a percentage, whether or not random assignment was possible, and whether the same people were tracked over time or two separate samples were compared. This guide first explains the general formula behind the calculation, then walks through how to apply it in AnalyZ Solutions.

01The general sample size formula

The most important decision this tool asks you to make is not a number at all. It is whether you are comparing two independent groups or the same group measured twice. This single choice changes the formula, the required inputs, and, usually significantly, the required sample size.

Two independent groups

Use this when comparing two separate samples: a treatment group against a comparison group, or two different people measured at baseline and again at endline. No individual is measured twice.

Each symbol in this formula plays a specific role.

• Z_1−α/2 (confidence level). The two-tailed critical value linked to how sure you want to be that a detected change is real, rather than random noise. At 95% confidence, this is 1.96.
• Z_1−β (power). The critical value linked to how likely you want to be to actually detect the change, if it is really there. At 80% power, this is approximately 0.84.
• σ (sigma), the expected standard deviation. A measure of how spread out individual values are around the average, in the same units as the outcome itself. A wider spread means any given difference between two group means is harder to distinguish from ordinary variation, so a larger σ increases the required sample.
• Δ (delta), the change to detect. The minimum difference between the two group means this calculation is built to reliably detect, in the same units as the outcome. A smaller Δ is harder to detect and requires a larger sample.

The factor of 2 in front of the formula exists because comparing two independent group means involves two separate sources of sampling variation, one from each group, rather than one. This is different from a proportion comparison, where a pooled variance term serves a similar purpose but is expressed differently, since the amount of variance in a proportion depends on the proportion itself in a way that a continuous outcome's variance does not.

Same group, before and after

Use this when the same individuals are measured at both time points, a genuine panel or cohort rather than two separate samples. Because each person acts as their own comparison, this design is statistically more efficient. It typically needs a meaningfully smaller sample to detect the same size of change.

This formula shares Z_1−α/2, Z_1−β, σ, and Δ with the independent groups version above, but introduces one new term.

• ρ (rho), the assumed correlation. A number between 0 and roughly 0.95 describing how consistent an individual's value tends to be between the two measurements. When someone scores high at baseline, do they also tend to score high at endline? A ρ of 0 means baseline tells you nothing about endline, and the paired formula reduces to the independent groups formula. A ρ closer to 1 means individuals tend to stay in a similar position relative to others, which sharply reduces the variance of the change score itself, and with it, the required sample.

The term 2σ²(1−ρ) is the variance of the difference between two correlated measurements on the same person. When ρ is 0, this equals 2σ², matching the independent groups formula exactly, since there is no relationship between the two measurements to exploit. As ρ rises toward 1, this term shrinks toward 0, reflecting that a person whose baseline and endline values are nearly identical relative to their peers contributes very little uncertainty to the estimated change.

Choosing your confidence level and power

Confidence level and power are judgment calls, not fixed rules, and they deserve as much thought as the effect size itself.

Confidence level reflects how willing you are to risk a false alarm, concluding a change happened when it actually did not. 95% is the standard default, meaning a 5% chance of that false alarm. A higher level, such as 99%, is worth considering before a costly or hard to reverse decision, such as expanding a programme nationally based on the result. A lower level, such as 90%, is sometimes acceptable for lower stakes, exploratory work.

Power reflects how willing you are to risk missing a real change entirely. 80% power is the conventional minimum, meaning a 20% chance of failing to detect a genuine change of the size specified. 90% power is worth considering for high stakes decisions, such as a funding renewal that depends on demonstrating measurable impact.

Choosing one-tailed or two-tailed

Both formulas above use Z_1−α/2, the two-tailed critical value, appropriate when you want to detect a change in either direction, an increase or a decrease. If your interest is specifically and only in one direction, for example only whether a programme raises average income, and a decrease genuinely does not need to be detected or reported, a one-tailed test using Z_1−α instead can be used. For 95% confidence, this changes the critical value from 1.96 to approximately 1.64, producing a smaller required sample for the same nominal confidence level.

Two-tailed is the safer, more defensible default. Choosing one-tailed specifically to reduce the required sample, without a genuine reason a change in the opposite direction is impossible or irrelevant, is a common way evaluations end up under-protected against being wrong.

Accounting for cluster sampling

If your sample will be drawn from clusters, villages, schools, or facilities, rather than a simple random sample, a design effect should be applied to either formula above. It defaults to 1, meaning no clustering effect assumed, and is always editable. See the dedicated guide on Design Effect and ICC for how to set this correctly.

The Change in Mean tool applies both formulas automatically based on the design you select, along with the one-tailed or two-tailed choice and the design effect.

1. Choose your design. Select Two independent groups or Same group, before and after. Read the short description AnalyZ Solutions displays under each option if you are unsure which applies to your study.
2. Set confidence and power. Enter both as numbers. Default values are 95% and 80%, respectively.
3. Enter the expected standard deviation and the change to detect. Both should be in the same units as the outcome you are measuring, for example both in points, both in local currency, or both in days.
4. For the paired design, enter an assumed correlation. If unsure, 0.5 is a reasonable default for most repeated measures indicators.
5. Set your expected non-response rate. This inflates the recruitment target so that, after accounting for people who do not respond, you still end up with enough completed pairs or completed groups.
6. Choose one-tailed or two-tailed. Two-tailed is selected by default and is appropriate for almost all evaluations.
7. Set a design effect, if sampling by cluster. Enter a value directly, or calculate one automatically from an intraclass correlation and average cluster size.
8. Calculate. The result shows the required sample, an interpretation of what it means, and the exact formula used, including which test type and design effect were applied.

02Worked example

A literacy programme wants to detect a change in average test score among the same cohort of learners, tested before and after a six month course. The expected standard deviation of scores is 12 points, the programme targets a 6 point improvement, and the assumed correlation between someone's before and after score is 0.5.

Before the non-response adjustment, the minimum statistical requirement is about 32 pairs. The 10% non-response rate inflates this to the recruitment target, so the study still ends with enough completed pairs after accounting for learners who cannot be reached or decline to participate at the second test.

The same 6 point change, calculated instead as two independent groups of learners rather than the same learners tracked twice, would require about 63 people per group before the non-response adjustment, roughly double, again illustrating the efficiency of paired designs whenever tracking the same individuals is feasible. That gap grows even larger as the assumed correlation ρ increases.

03Common mistakes to avoid

• Using the paired formula with two different samples. If different people were measured at baseline and endline, even if it is still described as before and after, that is an independent groups comparison. Applying the paired formula in that situation overstates the study's actual power.
• Guessing at the standard deviation. Because there is no conservative fallback value for σ the way there is for a proportion, an unrealistic guess is one of the most common reasons a continuous-outcome study ends up underpowered. Use pilot data or comparable published studies wherever possible.
• Choosing one-tailed testing purely to shrink the sample. One-tailed testing is only appropriate when a change in the opposite direction is genuinely not of interest and would not be reported.
• Mismatching units between the standard deviation and the change to detect. Both values must be expressed in the same units. Entering a standard deviation in one scale and a target change in another produces a meaningless result.
• Forgetting to inflate for non-response. The minimum statistical requirement and the actual recruitment target are different numbers, particularly for a paired design where losing a participant at either time point loses the whole pair.