How to compare percentages with a z-test for proportions

01What is the z-test for proportions?

Survey and study results are often stated as percentages: 62% of respondents are satisfied with a service, 48% of graduates found work within six months, 7% of items failed inspection. A percentage from a sample is an estimate, and it carries sampling uncertainty. The z-test for proportions helps you judge that uncertainty in two situations.

In the first, you compare one observed percentage with a target or reference value. Is 62% really above a 50% threshold, or could a sample of this size produce 62% when the true figure is 50%? In the second, you compare two percentages. Is the 68% in one group really different from the 41% in another?

Both rest on the same idea. If the true percentage in the population were a particular value, repeated samples of the same size would give percentages that scatter around it, and for reasonable sample sizes that scatter follows a bell-shaped normal curve. The test measures how many standard errors your observed result lies from the value being tested. A result far out in the tail would be rare if the null hypothesis were true, so it counts as evidence against it.

The null hypothesis is that the true proportion equals the target, or, in the two-sample version, that both groups share the same proportion.

02How the z statistic is calculated

One proportion against a target

Let n be the sample size, x the number of cases with the outcome of interest, and p₀ the target proportion.

Here p-hat is the observed proportion, and Φ is the area under the standard normal curve up to a given point. Doubling the upper tail area gives a two-tailed p-value, which counts extreme results in either direction. The standard error uses the target p₀ and not the observed proportion, because the test asks what would happen if the null hypothesis were true. In that situation, the spread of sample percentages is fixed by p₀.

Worked calculation. Suppose 149 of 240 respondents say they are satisfied, and the target is 50%.

1. Observed proportion
   149 divided by 240 gives 0.6208, or 62.1%.
2. Standard error
   The square root of (0.5 × 0.5 ÷ 240) is 0.0323.
3. z statistic
   (0.6208 - 0.5) ÷ 0.0323 gives 3.744. The observed percentage sits almost 3.7 standard errors above the target.
4. p-value
   The chance of a result at least this far out in either direction is about 0.0002, so the difference is significant at the 0.1% level.

The shaded areas hold the most extreme 5% of results. A z beyond about 1.96 in either direction is significant at the 5% level, and the observed value of 3.74 is far past that point.

The normal approximation behind the test works well when both n × p₀ and n × (1 - p₀) are at least 10. In this example both equal 120, so the condition is met comfortably.

Two proportions

For two samples, let x₁ out of n₁ and x₂ out of n₂ be the cases with the outcome.

The null hypothesis says both groups share one true proportion, so the best estimate of it combines all the cases. That combined value is the pooled proportion, and it replaces p₀ in the standard error.

Worked calculation. Suppose 136 of 200 urban residents own a smartphone, against 82 of 200 rural residents.

1. Observed proportions
   136 ÷ 200 = 0.68 and 82 ÷ 200 = 0.41, a difference of 0.27, or 27 percentage points.
2. Pooled proportion
   (136 + 82) ÷ 400 = 0.545.
3. Standard error
   The square root of (0.545 × 0.455 × (1/200 + 1/200)) is 0.0498.
4. z statistic
   0.27 ÷ 0.0498 gives 5.42, and the p-value is below 0.001.

There is a useful link with the chi-square test. Squaring this z gives 29.40, which is the chi-square statistic for the same two-by-two table without a continuity correction. The chi-square test in AnalyZ Solutions applies the correction for two-by-two tables and reports 28.32. The two routes lead to the same conclusion.

A confidence interval for a percentage

The test says whether a difference is likely to be real. An interval shows how precisely the percentage is known.

For the one-proportion example, the standard error is √(0.6208 × 0.3792 ÷ 240) = 0.0313, so the interval runs from 0.559 to 0.682. The true share of satisfied respondents is probably between 56% and 68%. This standard error uses the observed proportion, because no null hypothesis is assumed. The AnalyZ Solutions module reports the test result and not this interval, so calculate it by hand if you need it, or state the sample size next to the percentage so readers can judge the precision.

03How the z-test differs from the t-test

Both tests ask the same kind of question: how far is an observed value from a reference value, measured in standard errors? They differ in what is being measured and in where the standard error comes from.

The key difference is in the spread. For a yes or no outcome, once the proportion is known, the spread is known too. A group in which 62% say yes has a variance fixed at 0.62 × 0.38. For a numeric outcome such as income, the average says nothing about how spread out the values are, so the standard deviation has to be estimated from the data. That extra uncertainty is what the t curve allows for, and it matters most in small samples. A proportion has no such extra step, so the normal curve is used directly.

With large samples the two curves are almost identical, so the tests give similar numbers. For the 240 respondents above, coding satisfied as 1 and not satisfied as 0 and running a one-sample t-test against 0.5 gives t = 3.85, against z = 3.74. Both have a p-value of about 0.0002. The small gap arises because the t-test builds its standard error from the observed spread and the z-test builds it from the target. For percentages, the z-test is the principled choice.

04When to use it

Use the z-test for proportions when:

• The outcome is yes or no, or can be reduced to it, such as employed or not employed.
• Each respondent is counted once, so observations are independent.
• Both the expected successes and the expected failures are at least 10 under the tested value.
• The target was chosen before you looked at the results, or the two samples are separate.

Choose a different method when your situation changes:

• For a numeric outcome, use a one-sample or independent t-test.
• To test whether two categorical variables are related, or to compare more than two groups, use the chi-square test.
• To adjust a comparison for other variables such as age or income, use logistic regression.
• For very small samples, where the expected counts fall below 10, an exact binomial test is more reliable. AnalyZ Solutions does not include one, so report such results cautiously.

05How to run it in AnalyZ Solutions

Open Inferential Analysis, choose Compare Percentages, then Z-test for Proportion. Choose one of the two modes at the top.

One proportion against a target

1. Select the variable
   Pick the categorical variable that holds the outcome.
2. Choose the success value
   Pick the category that counts as a success, such as Yes or Employed. Value labels are shown next to the codes.
3. Enter the hypothesised proportion
   Type the target as a decimal between 0 and 1, for example 0.5 for 50%.
4. Click Run
   You receive the result table and a plain-language interpretation.

Two proportions

1. Select Variable 1 and its success value
   Pick the first outcome variable and the category that counts as a success.
2. Select Variable 2 and its success value
   Do the same for the second variable.
3. Click Run
   The output shows both percentages, the difference, and the significance level.

The two-proportion mode compares the share of successes in two separate variables, so it suits data where the two groups occupy separate columns, for example one column of results for one set of sites and another for a second set. If your data has a single outcome column and a separate column that identifies the group, use the chi-square test on the two variables. For a two-by-two table, it answers the same question.

06Interpreting the result

The figures below are invented to show the steps, and they follow the results tables in AnalyZ Solutions, with a few columns left out to save space. Suppose 149 of 240 respondents say they are satisfied with a service, and the target is that at least half are satisfied. This example uses the one-proportion mode.

Reading it step by step

1. Read the observed percentage
   The observed proportion is 0.6208, which is 62.1%. The target is 0.5, so the sample sits 12.1 percentage points above it.
2. Check the sample size
   There are 149 successes and 91 failures, both well above 10, so the normal approximation behind the z-test is sound.
3. Check significance
   The z value of 3.7439 is far from zero and the p-value is below 0.001 with three stars. A sample of 240 would almost never show 62% if the true figure were 50%, so the gap is significant at the 0.1% level.
4. Judge whether the gap matters
   Twelve points is a clear margin on a percentage, and it is the number to report because everyone can read it. The test itself does not give a confidence interval, so say how many people were surveyed next to the percentage.

In plain words

About 62% of the 240 respondents were satisfied with the service, above the 50% target. The gap is significant at the 0.1% level. In two-proportion mode, the reading works the same way: report the difference in percentage points from the Diff column, and use the z value and p-value to say whether it is real.

07Common mistakes

Ignoring the sample size. A percentage from 20 respondents is far less certain than one from 2,000, even when the number looks the same.

Comparing overlapping groups. The two-proportion test assumes the two samples are separate. Using it on two questions answered by the same respondents violates this.

Ignoring clustering. If respondents were sampled in schools or workplaces, the test understates the uncertainty.

Reporting the z statistic instead of the difference. Readers understand percentage points far better than z values.

Treating a significant result as an important one. A one-point gap can be significant in a very large sample. Judge the size of the difference as well as its significance.