How to Calculate Sample Size to Detect a Change in Proportion

A large share of programme evaluations come down to one question: did a percentage move? Did employment go up? Did contraceptive use increase? Did school attendance improve? This tool is built for exactly that question, whether or not random assignment was possible, and whether the same people were tracked over time or two separate samples were compared. This guide first explains the general formula behind the calculation, then walks through how to apply it in AnalyZ Solutions.

01The general sample size formula

The most important decision this tool asks you to make is not a number at all. It is whether you are comparing two independent groups or the same group measured twice. This single choice changes the formula, the required inputs, and, usually significantly, the required sample size.

Two independent groups

Use this when comparing two separate samples: a treatment group against a comparison group, or two different people surveyed at baseline and again at endline, sometimes called a repeated cross-section. No individual is measured twice.

Each symbol in this formula plays a specific role.

• Z_1−α/2 (confidence level). The two-tailed critical value linked to how sure you want to be that a detected change is real, rather than random noise. At 95% confidence, this is 1.96.
• Z_1−β (power). The critical value linked to how likely you want to be to actually detect the change, if it is really there. At 80% power, this is approximately 0.84.
• p₁ (baseline proportion). Your best estimate of the proportion before the period you are studying, for example the percentage employed before a training programme begins.
• p₂ (endline proportion). The proportion you expect, or need, to see after the period you are studying. The gap between p₁ and p₂, often written δ (delta), is the minimum change this calculation is built to reliably detect.
• p̄ and q̄. p̄ is the average of p₁ and p₂. q̄ is 1 minus p̄. These appear together because the formula needs a single pooled estimate of variance under the assumption that, if there were truly no difference, both groups would share this common proportion.
• q₁ and q₂. Simply 1 minus p₁ and 1 minus p₂. These capture the actual variance implied by each group's own proportion, separately from the pooled p̄q̄ term.

Same group, before and after

Use this when the same individuals are measured at both time points, a genuine panel or cohort rather than two separate samples. Because each person acts as their own comparison, this design is statistically more efficient. It typically needs a meaningfully smaller sample to detect the same size of change.

This formula shares p₁, p₂, q₁, and q₂ with the independent groups version above, but introduces one new term.

• ρ (rho), the assumed correlation. A number between 0 and roughly 0.95 describing how consistent individuals tend to be between the two measurements. A ρ of 0 would mean someone's baseline status tells you nothing about their endline status, and the paired formula collapses toward the independent groups formula. A ρ closer to 1 means individuals tend to stay in the same state, which sharply reduces the variance of the change score and, in turn, the required sample.

Notice that the entire bracketed term, [p₁q₁+p₂q₂−2ρ√(p₁q₁p₂q₂)], is smaller than the independent groups term whenever ρ is positive. This is the mathematical expression of the intuition that tracking the same people over time captures information a fresh, unrelated sample cannot.

Choosing your confidence level and power

Confidence level and power are judgment calls, not fixed rules, and they deserve as much thought as the effect size itself.

Confidence level reflects how willing you are to risk a false alarm, concluding a change happened when it actually did not. 95% is the standard default across social research and evaluation, meaning a 5% chance of that false alarm. A higher confidence level, such as 99%, is worth considering when acting on a wrong conclusion would be costly or hard to reverse, for example before scaling a programme nationally. A lower level, such as 90%, is sometimes used for lower stakes, exploratory work where a smaller sample is preferable and an occasional false alarm is an acceptable trade-off.

Power reflects how willing you are to risk missing a real change entirely. 80% power is the conventional minimum, meaning a 20% chance of failing to detect a genuine change of the size specified. 90% power is worth considering for high stakes decisions, such as a funding renewal that depends on demonstrating impact, where missing a real effect would be especially damaging.

Choosing one-tailed or two-tailed

Both formulas above use Z_1−α/2, the two-tailed critical value, appropriate when you want to detect a change in either direction, an increase or a decrease. If your interest is specifically and only in one direction, for example only whether a programme increases employment, and a decrease genuinely does not need to be detected or reported, a one-tailed test using Z_1−α instead can be used. For 95% confidence, this changes the critical value from 1.96 to approximately 1.64, producing a smaller required sample for the same nominal confidence level.

Accounting for cluster sampling

If your sample will be drawn from clusters, villages, schools, or facilities, rather than a simple random sample, a design effect should be applied to either formula above. It defaults to 1, meaning no clustering effect assumed, and is always editable. See the dedicated guide on Design Effect and ICC for how to set this correctly.

The Change in Proportion tool applies both formulas automatically based on the design you select, along with the one-tailed or two-tailed choice and the design effect.

1. Choose your design. Select Two independent groups or Same group, before and after. Read the short description AnalyZ Solutions displays under each option if you are unsure which applies to your study.
2. Set confidence and power. Enter both as numbers, not from a fixed list, so any value is possible. 95% confidence and 80% power are reasonable defaults if you have no specific reason to depart from them.
3. Enter baseline and expected endline proportions. Base these on prior data, a pilot, or, where genuinely nothing is known, a plausible planning assumption. The gap between them is the effect size the study is built to detect.
4. For the paired design, enter an assumed correlation. If unsure, 0.5 is a reasonable default for most repeated measures indicators. Higher values assume more consistency between someone's baseline and endline status.
5. Choose one-tailed or two-tailed. Two-tailed is selected by default and is appropriate for almost all evaluations.
6. Set a design effect, if sampling by cluster. Enter a value directly, or calculate one automatically from an intraclass correlation and average cluster size.
7. Calculate. The result shows the required sample, an interpretation of what it means, and the exact formula used, including which test type and design effect were applied.

02Worked example

A vocational training programme wants to show a rise in formal employment among the same cohort of graduates, tracked before enrolment and twelve months after completion. Baseline employment is 25%, the programme targets 40%, and the expected correlation between someone's baseline and follow-up employment status is 0.4.

For comparison, sizing the identical 25% to 40% change as two independent groups, rather than the same cohort tracked twice, would require roughly 133 people per group at the same confidence and power, close to 50% more total participants than the paired design. This gap is entirely the value of tracking individuals rather than sampling fresh at each round, and it grows larger still as the assumed correlation ρ increases.

03Common mistakes to avoid

• Using the paired formula with two different samples. If different people were surveyed at baseline and endline, even if it is still described as before and after, that is an independent groups comparison. Applying the paired formula in that situation overstates the study's actual power.
• Choosing one-tailed testing purely to shrink the sample. One-tailed testing is only appropriate when a change in the opposite direction is genuinely not of interest and would not be reported. Used otherwise, it understates the true risk of a false positive.
• Guessing at ρ without basis. An assumed correlation that is too optimistic understates the required sample, and one that is too conservative wastes budget. Where a prior round of the same or a similar panel exists, calculate ρ from it rather than assuming.
• Sizing the study around an unrealistically large expected change. A programme that in practice shifts a proportion by 5 percentage points, sized instead around a hoped-for 15 point shift, will very likely be underpowered to detect its own real effect.