How to Audit Your Data Before Analysis

Most data problems are invisible until they distort a result. Missing values reduce your effective sample without warning. Numeric codes masquerade as real responses. A column that never changes passes silently through every analysis while contributing nothing. Certain enumerators may have collected data carelessly, or not at all. A data quality audit finds all of these before any analysis begins.

01What is data quality, and why does it matter?

Data quality refers to how fit a dataset is for the analysis you intend to run. A dataset can be large, well-structured, and carefully collected and still contain problems that distort results: missing values that quietly reduce the sample, numeric codes masquerading as real responses, columns where every row holds the same value, or individual entries so extreme they are almost certainly errors.

None of these problems are dramatic. They do not crash your software or produce obvious error messages. They simply make your results less accurate or less trustworthy in ways that are hard to detect once analysis is underway. A mean on a variable with 40% missing data describes only the 60% who responded. A regression that includes a constant variable will fail. A frequency table that treats 999 as a real response distorts every percentage it contains.

The answer is to examine the data before analysing it, systematically, across all variables, using a standard set of checks. That is what a data quality audit does.

02The types of problem a quality check finds

The Data Quality module in AnalyZ Solutions is organised into four tabs, each targeting a distinct category of data problem. You run each tab independently, in any order, and only configure the checks that are relevant to your dataset.

Missing values

A missing value is any cell where no response was recorded. Missingness happens for many reasons: a question was skipped, an enumerator left a field blank, a respondent declined to answer, or a merge left some rows without a match. The key distinction is between missing completely at random (MCAR), where gaps are spread across the dataset without pattern, and missing not at random (MNAR), where gaps are concentrated in particular groups or enumerators. The latter is far more problematic, because the missing cases are systematically different from the recorded ones, which means any analysis on the observed data may be biased.

Missing value codes are a specific trap in survey data. Tools like KoBo Toolbox and ODK sometimes store "not applicable" or "refused" responses as numeric codes such as 99, 999, or -1, rather than true blanks. If these codes are not defined as missing before analysis, they appear in every calculation as real values, inflating means, distorting distributions, and understating the true missing count.

Constant and low-variance variables

A constant variable is a column where every row has the same value. It carries no information and cannot explain anything. In most analytical methods it causes errors: regression algorithms fail when a predictor has zero variance, and factor analysis cannot extract a factor from a variable with no variability.

A low-variance variable is a step below constant: not entirely uniform, but dominated by a single category (95% or more of valid responses fall in one category). These variables can technically be included in analysis, but the statistical tests that depend on them will be unreliable because there are too few observations in the minority category to estimate effects accurately.

Statistical outliers

An outlier is a value that sits far from the rest of the distribution. AnalyZ Solutions uses the standard definition: any value more than three standard deviations from the mean is flagged. Outliers are not automatically errors. A household income ten times the average may be a genuine extreme case. But an age of 150 or a weight of 9,999 is almost certainly a data entry error. The quality check flags potential outliers; it does not remove them. Investigation always comes before any decision to recode or drop a value.

Duplicate rows

A duplicate row is a record that is completely identical to at least one other record across all columns. In survey data, duplicate rows commonly occur when a submission is uploaded twice from the field, when a KoBo or ODK form is accidentally submitted more than once, or when two datasets are merged without deduplication. Duplicates inflate your effective sample size and give certain observations disproportionate weight in every analysis.

Range violations

A range violation is a value that falls outside a plausible or expected boundary for that variable. For example, a value of 150 for an age variable, a negative value for a count, or a percentage above 100 are all implausible. AnalyZ Solutions checks for these in two ways: through user-defined rules (you specify the minimum and maximum for any variable you choose) and through auto-detection, which applies heuristics based on variable names and labels, flagging negative values in variables that suggest counts, ages, or amounts, and values above 100 in variables that appear to be proportions or percentages.

Digit preference

Digit preference, also called heaping, is the tendency for respondents or enumerators to favour round numbers when estimating or reporting values. In a dataset without heaping, the last digit of a numeric variable should be roughly uniformly distributed across 0 to 9. When enumerators are guessing or rounding, digit 0 and digit 5 tend to appear far more often than the others. Significant heaping on any digit is a signal that reported values may not be precise, and in more serious cases, that data may have been fabricated.

Enumerator patterns

An enumerator is a field worker who administers survey interviews. In large multi-enumerator surveys, data quality problems are rarely distributed evenly. They tend to concentrate in the work of specific enumerators who may have misunderstood questions, taken shortcuts, or in the worst case fabricated responses. Enumerator pattern analysis examines each enumerator's work as a group and flags statistical anomalies: unusually high rates of missing data, suspiciously short interview durations, or straight-lining.

Straight-lining is the practice of giving the same response to every item in a set of scale questions. For example, answering 3, 3, 3, 3, 3 across a five-item Likert scale. It is a strong signal of inattentive responding or interview fabrication. AnalyZ Solutions checks for straight-lining within named scale groups, so you can define separate groups for different scales (for example, a wellbeing scale and a food security scale) and check each independently. A respondent is flagged as a straight-liner if their standard deviation across all items in any single group equals zero.

Open Descriptive Analysis and select Data Quality. The module has a Configure panel at the top and four independent tabs below it. Each tab has its own Run button, and you run only the checks you need.

Setting up the Configure panel

Expand Configure optional checks before running any tab. This is where you set up the checks that go beyond the basic variable-level summary.

1. Enumerator ID variable. Select the column that identifies who collected each record. Required for the Enumerator Patterns tab and for the per-enumerator duplicate breakdown in the Duplicate Rows tab.
2. Interview start and end time. Select the timestamp columns that record when each interview began and ended. Used to calculate interview duration per enumerator. KoBo Toolbox and ODK Central export these automatically.
3. Scale groups for straight-lining. Add one group per Likert or rating scale in your questionnaire. Give each group a label (for example, "Wellbeing scale" or "Food security index") and select all the variables that belong to it. Each group is checked independently; a respondent is flagged if their responses within any single group have zero variance.
4. Custom range rules. For each variable where you know the valid range, add a rule specifying the minimum, maximum, or both; you can also enable Auto-detect implausible values to flag common implausibles automatically based on variable names.

Tab 1: Overview

Run this first. It checks every variable for missing values, duplicate entries within each column, outliers, constants, and low-variance variables, and produces a one-row-per-variable summary table. No configuration is required.

Tab 2: Duplicate Rows

Finds rows that are completely identical across all columns. Results include the total number of duplicate rows, how many distinct duplicate groups exist (a group is a set of two or more identical records), and a preview table with all duplicate records highlighted by group. If an enumerator variable is configured, a per-enumerator breakdown is also shown.

Tab 3: Value Issues

Shows two sub-sections. Range violations lists every variable where a rule was applied (either custom or auto-detected), the number of violations, their percentage of valid cases, and up to ten sample values so you can inspect the nature of the problem. Digit preference shows a bar chart for each numeric variable with sufficient whole-number values, with the frequency of each last digit (0 to 9) plotted against the expected frequency if responses were uniform. Bars shown in amber exceed twice the expected frequency.

Tab 4: Enumerator Patterns

Requires an enumerator variable to be set in Configure; the tab is disabled until one is selected. Produces a per-enumerator summary table with average missing rate, straight-lining percentage (if scale groups are configured), and median interview duration (if start and end timestamps are configured). Enumerators with anomalous values are flagged automatically. A second panel below the table expands the detail for each flagged enumerator, including which scale group triggered the straight-lining flag.

03How to read the output

Overview tab output

At the top, four summary cards give an immediate picture: total variables checked, total missing cells across the dataset, the overall percentage of missing data, and the count of variables with at least one flagged issue. Below the cards, the variable-level table shows one row per variable:

Issue flags and what they mean

• OK: no issues detected. The variable is clean and ready for analysis.
• High missing (>30%): more than 30% of rows have no value. Investigate whether missingness is structural (a question shown only to a subgroup) or random. If random, the variable may need to be dropped or imputed before analysis.
• Moderate missing (10–30%): check whether gaps are concentrated in particular enumerators, regions, or collection rounds. Clustered missingness biases results even at moderate rates.
• Constant: every valid response is the same value. The variable contributes nothing analytically and should be removed.
• Low variance: one category accounts for 95% or more of valid responses. The variable can technically be included in analysis, but statistical tests will be unreliable.
• Outliers: one or more values exceed three standard deviations from the mean. Inspect before deciding to recode or retain.

Enumerator Patterns tab output

The anomaly flagging logic uses the following thresholds:

• High missing: average missing rate across all variables exceeds 20% for that enumerator's records.
• Straight-lining: more than 10% of that enumerator's interviews contain a row where all responses within any single scale group are identical.
• Short duration: the enumerator's median interview duration is a statistical outlier on the low end, defined as below Q1 minus 1.5 times the interquartile range across all enumerators. This identifies enumerators who consistently complete interviews significantly faster than their peers.

04What to do when you find problems

A data quality audit identifies problems. It does not fix them. Each type of finding calls for a different response, and several other modules in AnalyZ Solutions are directly useful for acting on what the audit reveals.

Missing values

The right response to missing data depends on why it is missing and how much of it there is. For variables with very high missingness (>50%), ignoring them in the analysis is one of the simplest approaches. You can choose to delete such variables, if you want to prevent other users of data from using them. For moderate missingness in a variable that is central to your analysis, imputation may be warranted. Review the distribution of missingness across enumerators and collection periods before deciding; concentrated missingness is a field problem, not a statistical one.

Duplicate rows

Once you have confirmed which rows are genuine duplicates (not legitimate repeat interviews), remove them before any analysis. Keeping duplicate rows inflates your sample size and gives those observations double weight in every frequency, mean, and model.

Range violations

Values outside a plausible range are either data entry errors or genuine extreme cases. Inspect each one individually before deciding what to do. Common responses are: correcting the value if the true value is known from source documents, recoding to missing if the error cannot be corrected, or flagging the observation for exclusion from specific analyses.

Low-variance variables

Low-variance variables should be reviewed: if the imbalance reflects a genuine feature of the population, you may choose to retain the variable with a note; if it reflects a data collection problem, removal or recoding is appropriate.

Outliers

Never remove an outlier without inspecting it first. Open the raw data in Data View, sort the variable in question, and look at the actual values. An age of 150 is almost certainly a data entry error. A household income of ten times the median may be a legitimate extreme case worth retaining. If you do recode an outlier to missing, document the decision and the threshold you applied so others can reproduce it.

Enumerator anomalies

An enumerator flagged for high missing rates, short durations, or straight-lining warrants investigation, not immediate action. Review the raw records for that enumerator in Data View, filtered to their submissions. If the pattern is consistent across multiple variables and interviews, escalate to field supervision. If data fabrication is suspected, the affected records may need to be excluded from analysis entirely and noted in the methods section of your report.

Digit preference

Heaping on round numbers (0 and 5) is common in self-reported continuous variables and does not always indicate a serious problem. Severe heaping, where one or two digits account for a majority of responses, is a stronger signal. In age and income data, heaping is expected and usually handled by working with ranges or rounded values rather than precise point estimates. In health or anthropometric data, heaping can indicate that measurements were estimated rather than taken, which has implications for precision.

05Common mistakes to watch for

• Skipping the check entirely. Starting with frequencies or summaries before auditing is the most common mistake. Problems found after analysis has begun are slower and harder to fix.
• Treating missing as zero. A blank cell and a zero are not the same thing. A missing income is not an income of zero.
• Removing all flagged outliers without investigation. Deleting every value beyond three standard deviations is as risky as ignoring them. Each flagged value needs to be looked at individually.
• Not defining missing value codes first. Numeric codes like 99 or 999 appear as real responses in the report unless you define them in Data Management beforehand.
• Confusing within-column duplicates with duplicate rows. The Duplicates column in the Overview tab counts repeated values within a single variable, which is expected for categorical variables. The Duplicate Rows tab checks whether entire records are identical, which is the more serious concern.
• Only running the check once. Re-run after merges, appends, and major cleaning steps. The check takes seconds and the dataset changes each time you modify it.
• Treating enumerator flags as proof of wrongdoing. An anomaly flag is a signal to investigate, not a finding. Short durations could reflect a genuinely efficient interviewer. High missing rates could reflect a legitimately hard-to-reach subgroup. Always review the raw data before drawing conclusions.