How to Perform a Chi-Square Test

Survey data is full of questions that look simple on the surface: does programme participation vary by gender? Does satisfaction differ between districts? Do trained and untrained groups respond differently? These are questions about whether two categorical variables are related — and the chi-square test, run through a cross-tabulation, is the standard way to answer them.

01What is the chi-square test?

The chi-square test is a way of asking whether two categorical variables are related to each other, or whether their distributions are independent. Two variables are independent when knowing someone's value on one tells you nothing about their value on the other. They are related when the distribution of one shifts depending on the value of the other.

The test works by comparing what you actually observed in the data with what you would expect to see if the two variables were completely unrelated. Those expected values are calculated from the row and column totals alone — they represent the counts that would fill each cell if the variables had no connection at all. If the observed and expected counts are close, the data is consistent with independence. If they diverge substantially, there is evidence of a relationship.

The chi-square statistic summarises the total amount of that divergence across all cells in the table. The larger it is, the further the data is from what independence would predict. The p-value then converts that statistic into a probability: the chance of seeing a difference at least this large if the variables were truly unrelated in the wider population. A small p-value means that chance alone is an unlikely explanation for the pattern you observed.

The chi-square test does not describe what the relationship is, how strong it is, or which direction it runs. It only answers whether the evidence is strong enough to conclude that a relationship exists at all. Understanding the nature and size of the relationship requires reading the cross-tabulation table alongside the test result.

02What is cross-tabulation, and how does it relate?

A cross-tabulation — also called a contingency table or crosstab — is the format in which the chi-square test is presented and read. It arranges the two variables in a grid: the categories of one variable form the rows, and the categories of the other form the columns. Each cell in the grid shows how many respondents fall into that particular combination of values.

The table makes the pattern visible in a way the test statistic alone cannot. Where a chi-square value just gives you a number, the cross-tabulation shows you where observed and expected counts diverge — which groups are over-represented, which are under-represented, and what the actual distributions look like across each category. The two always go together: the table shows you what is happening; the test tells you whether what you see is likely to reflect a real pattern or could plausibly be noise from sampling.

For example, if you cross-tabulate programme participation (yes / no) by gender (male / female), the table shows you the participation rate for each gender side by side. The chi-square test then tells you whether the difference between those rates is large enough to be taken seriously given your sample size.

03What is Cramér's V?

The chi-square statistic has one important limitation: it grows with sample size. A relationship of exactly the same strength will produce a larger chi-square value in a dataset of 5,000 respondents than in one of 200. This means the statistic alone cannot tell you how strong the relationship is — only that it is detectable given the sample you have.

Cramér's V solves this by converting the chi-square statistic into a standardised measure of association that is independent of sample size. It ranges from 0 to 1, where 0 means the two variables are completely unrelated and 1 means knowing one variable perfectly predicts the other.

As a practical guide for interpreting Cramér's V in social and evaluation research:

• Below 0.10 — negligible association. The relationship may be statistically detectable but is unlikely to be practically meaningful.
• 0.10 to 0.19 — small association. Worth noting but not strong enough to drive conclusions on its own.
• 0.20 to 0.29 — moderate association. A meaningful pattern that is worth describing and reporting.
• 0.30 and above — strong association. The two variables have a clear and substantial relationship.

These thresholds are not rigid rules — what counts as meaningful depends on the context. Use them as a starting point for interpretation, not as a mechanical cutoff.

04A worked example

Suppose a baseline survey of 200 households records whether each household is enrolled in a nutrition programme (yes or no) and the respondent's education level (primary or secondary). The cross-tabulation looks like this:

Step 1: Calculate expected counts

If education and enrolment were completely unrelated, each cell would contain a count proportional to its row and column totals. The formula for any cell is:

Expected = (Row total × Column total) ÷ Grand total

Applying this to each cell:

• Primary, Enrolled: (100 × 90) ÷ 200 = 45
• Primary, Not enrolled: (100 × 110) ÷ 200 = 55
• Secondary, Enrolled: (100 × 90) ÷ 200 = 45
• Secondary, Not enrolled: (100 × 110) ÷ 200 = 55

These expected counts represent what the table would look like if education had no bearing on enrolment. Compare them with the observed counts: primary-educated households are enrolled at 30%, while secondary-educated households are enrolled at 60%. The pattern clearly diverges from what independence would produce.

Step 2: Calculate the chi-square statistic

The chi-square statistic sums the squared difference between observed and expected counts, divided by the expected count, across all cells:

χ² = Σ (Observed − Expected)² ÷ Expected

• Primary, Enrolled: (30 − 45)² ÷ 45 = 225 ÷ 45 = 5.00
• Primary, Not enrolled: (70 − 55)² ÷ 55 = 225 ÷ 55 = 4.09
• Secondary, Enrolled: (60 − 45)² ÷ 45 = 225 ÷ 45 = 5.00
• Secondary, Not enrolled: (40 − 55)² ÷ 55 = 225 ÷ 55 = 4.09

χ² = 5.00 + 4.09 + 5.00 + 4.09 = 18.18

With 1 degree of freedom ([rows − 1] × [columns − 1] = 1 × 1 = 1), a chi-square of 18.18 gives a p-value well below 0.001. The result is highly statistically significant — the pattern is very unlikely to have arisen by chance.

Step 3: Calculate Cramér's V

To understand the strength of the relationship, we calculate Cramér's V. For a 2 × 2 table, the formula simplifies to:

V = √(χ² ÷ n) = √(18.18 ÷ 200) = √0.0909 = 0.30

A Cramér's V of 0.30 indicates a strong association. Secondary-educated households are substantially more likely to be enrolled — and the size of that difference is large enough to be practically meaningful, not just statistically detectable.

05How to do it in AnalyZ Solutions

Open Descriptive Analysis and select Cross-Tabulate.

1. Select the variable on the side (rows). Choose the variable whose distribution you want to explain or describe. Its distinct values will appear as row labels down the left side of the table.
Layout preview
As soon as you select a variable, a table layout preview appears below the Generate button. It shows a skeleton of the output — the column variable as header, the row variable down the side — so you can confirm the structure is correct before running. The preview is replaced by the real table when you click Generate. If you change your variable selection after generating, the preview reappears automatically to reflect the new selection.
2. Select the variable at the top (columns). Choose the grouping variable — the one you want to compare across. Its distinct values will appear as column headers across the top.
3. Choose the percentage type. Select Row %, Column %, Total %, or Counts only. Column % is the most common choice in evaluation reports.
4. Include missing values (Optional). Tick Include missing values to add a Missing category for either variable, so non-response is visible rather than silently excluded.
5. Apply a subsample filter (Optional). Apply filter to restrict the analysis to a subset of rows.
6. Click Generate. The table and test statistics appear immediately.

06How to read the output

The output has two parts: the cross-tabulation table and the test statistics below it.

Reading the table

The table shows the values of one variable as row labels down the side, and the values of the other as column headers across the top. Each cell shows a count and, depending on the percentage option you chose, a percentage. The last row and column are marginal totals.

The most useful percentage direction depends on which variable is doing the grouping:

• Column % — divides each cell by the column total. Use this when the column variable is the grouping factor. Column percentages let you compare the row variable's distribution across each column group by reading down each column.
• Row % — divides each cell by the row total. Use this when the row variable is the grouping factor. Row percentages show what proportion of each row category falls into each column category.
• Total % — divides each cell by the overall total. Useful when you want to show the share of the whole sample that each cell combination represents.
• Counts only — raw numbers with no percentages. Useful for small tables or when counts are what you need to report.

Reading the test statistics

Three statistics appear below the table. Read all three together for a complete picture of the result.

• Chi-square (χ²) — the total divergence between observed and expected cell counts. A value near zero means the data is consistent with independence. A large value means observed and expected counts diverge substantially.
• p-value — the probability of observing a chi-square value at least this large if the variables were truly independent. Below 0.05 is conventionally taken as evidence of a real relationship; above 0.05 means the data does not provide strong enough evidence to rule out chance.
• Cramér's V — the strength of the association on a 0 to 1 scale, independent of sample size. Use the interpretation guide in section 03 to judge whether the value is practically meaningful.

07Common mistakes to watch for

• Reading the wrong percentage direction. If you want to compare how a variable differs across groups defined by the column variable, use column percentages and read down each column. Reading across rows answers a different question.
• Treating a significant p-value as evidence of an important finding. A p-value below 0.05 tells you the relationship is unlikely to be zero. It does not tell you the relationship is large, meaningful, or worth acting on. Always report Cramér's V alongside the p-value.
• Ignoring the direction and pattern in the table. A significant chi-square result tells you a relationship exists — it does not tell you what it looks like. Always read the percentage columns or rows to describe which groups differ and in what direction.
• Including too many categories. Tables with many row and column categories will have cells with very small expected counts, making the chi-square test unreliable. Combine rare categories using Recode Variable before running.
• Using chi-square for numeric variables. If a variable has many distinct numeric values, it belongs in Summary Statistics or the Inferential module, not in a cross-tabulation.
• Forgetting the independence assumption. Chi-square requires each respondent to appear only once in the table. Panel data, repeated measures, or matched pairs violate this assumption and require different tests.

08Export and next steps

Results can be exported as CSV, Excel, or Word. The Word export includes the table, the chi-square statistics, and a plain-English interpretation of the result. Use Add to AnalyZense to include the result in a project, or Add to Canvas to place it on the Data Canvas.

After cross-tabulation, a natural next step is to explore whether the relationship holds within subgroups. Re-run the crosstab with a subsample filter to check, for example, whether the education-by-enrolment pattern looks the same in urban and rural areas, or across different programme types.