How to find out which survey questions belong together

01What is exploratory factor analysis?

When you write a questionnaire you usually have themes in mind, such as questions about the work itself and questions about the manager. Exploratory factor analysis, or EFA, checks from the answers people actually gave whether the questions fall into those groups. Questions that people answer in a similar way, so that those who agree with one tend to agree with another, are treated as measuring the same theme. Each theme is called a factor. You never see a factor directly. You infer it from the pattern in the answers.

An example

You survey 300 health workers with ten statements about their jobs, each rated from 1 to 5. You wrote them with three ideas in mind: how meaningful the work feels, how good the supervision is, and how manageable the workload is. Before you add the items into three subscale scores, you want evidence that the answers really fall into three groups, and that each statement sits in the group you intended. EFA gives you that evidence from the data.

Key terms

• Factor. A theme behind a group of questions. It is not measured directly. It shows up in how people answer several questions in the same way.
• Loading. How closely a question is tied to a factor. Ignoring the sign, a bigger number means a closer tie.
• Communality. How much of the variation in a question's answers the factors explain together.
• Uniqueness. What the factors do not explain. It is the part that belongs to the question alone, plus measurement error.

EFA compared with PCA

What you get out of it

1. Suitability checks
   KMO and Bartlett's test, which show whether the items are correlated enough for a factor analysis.
2. Factor loadings
   A table and heatmap of how strongly each item loads on each factor.
3. Variance and communalities
   How much each factor explains, and how well the factors account for each item.
4. Factor correlations
   For an oblique rotation, how strongly the factors are related to each other.

02When to use it

Use EFA when you do not yet know the structure of a set of items, or when you want to check the structure before building scales. Typical cases are developing a new questionnaire, adapting one to a new language or population, and finding items that fit badly and could be removed.

If you already have a firm theory about which item belongs to which factor, and want to test how well that structure fits, the right tool is confirmatory factor analysis, which AnalyZ Solutions does not offer. EFA is also not the choice when you only want to condense variables. For that, use principal component analysis. Once you know which items form a factor, check each group with Cronbach's alpha.

How much data you need

A common minimum is five respondents for each item, so ten items need at least 50 people. Results are steadier with 100 or more, and with 200 or more when the loadings are modest. The items should be numeric ratings with at least five response options. Yes or no items distort the correlations that EFA relies on.

03The factor model

EFA describes each standardised item as a weighted sum of the factors plus a part that belongs to the item alone.

For the item I plan to stay here in the worked example below, the loadings are 0.463, 0.258 and 0.073, so the communality is 0.463² + 0.258² + 0.073² = 0.286. The factors explain 28.6% of the variation in that item, and the remaining 71.4% is unique to it.

The variance explained by a factor is the sum of the squared loadings of all items on it, called the sum of squared loadings, divided by the number of items. It shows how much of the total variation among the items that factor accounts for.

Are the items suited to factor analysis?

The module reports two checks before you read any loadings.

Bartlett's test of sphericity asks whether the correlation matrix differs from a matrix in which no items are correlated. If it does not, there is nothing to factor. The test uses the determinant of the correlation matrix, written |R|, with p items and N people.

With a large sample it is almost always significant, so passing it is a low bar. The module shows it in a green banner when p is below 0.05, and in grey otherwise.

The KMO measure compares ordinary correlations with partial correlations, and runs from 0 to 1. Values above about 0.60 are usually accepted, and the module labels 0.70 to 0.79 as middling, 0.80 to 0.89 as meritorious and 0.90 and above as marvellous. Below 0.50 is unacceptable.

04Extraction, number of factors and rotation

Extraction method

The module offers two ways to extract the first solution.

• Principal axis factoring (PAF) is the default. It starts with an estimate of each item's communality, puts those estimates on the diagonal of the correlation matrix, extracts the factors, and then re-estimates the communalities from the result. It repeats up to 25 times until the estimates settle. Because it uses only shared variation, it follows the logic of factor analysis.
• Principal components uses all the variation in each item, as PCA does. Communalities come out higher, and the result is closer to a PCA than to a true factor model.

For most questionnaire work, PAF is the better choice.

How many factors

You choose the number, and it must be smaller than the number of items. The usual guides are the same as for PCA: how many eigenvalues of the correlation matrix are above 1, where the scree plot bends, and whether the factors make sense in words. The EFA module does not draw a scree plot. To see one, run a PCA on the same items. Theory counts too. If you designed the questionnaire around three ideas, a three-factor solution is the first one to test.

Rotation

An unrotated solution puts most of the weight on the first factor, so many items load on it and the picture is blurred. Rotation turns the factors to make each item load strongly on one and weakly on the others, which is called simple structure. It changes how the variation is shared out among the factors. It does not change the communalities.

• Varimax (orthogonal) keeps the factors uncorrelated. It is simple and gives clear results, and it is the module's default.
• Oblimin (oblique) lets the factors correlate. Many scales in the social sciences measure related ideas, so this is often the more realistic choice. The module adds a Factor correlations tab for this option.
• None (unrotated) shows the raw solution.

A practical rule is to try Oblimin first. If the factor correlations are all low, Varimax would give much the same picture. The module comments on this: a correlation above 0.32 between two factors suggests that an oblique rotation is appropriate.

05Reading the output

The top of the results shows the Bartlett banner, then the number of factors extracted, the percentage of variance explained and the KMO value with its label. Up to four tabs follow.

Factor loadings

A heatmap shows the loading of every item on every factor, shaded by size. The same display options as in PCA apply. Dominant factor lists each item under the factor where it loads most strongly, and By threshold lists it under every factor where its loading reaches the minimum, which is 0.35 to begin with.

A well-behaved item loads 0.40 or more on one factor and below 0.30 on all the others. An item that loads 0.30 or more on two factors is a cross-loader, and the interpretation text names it as ambiguous.

Variance explained

The table gives the sum of squared loadings, the percentage of variance and the cumulative percentage for each factor, with a bar chart. These percentages are lower than the ones PCA gives for the same items, because only shared variation is counted. The module calls 60% or more a reasonable result, and 40% to 60% a moderate one.

Communalities

Communality and uniqueness are listed for each item. Values below 0.30 are highlighted. A low communality means that the factors explain little of the item.

Factor correlations

This tab appears only with Oblimin. It lists the correlation between each pair of factors.

06How to run it in AnalyZ Solutions

Open Scale & Reliability, then choose Exploratory Factor Analysis.

1. Select the variables
   Choose three or more numeric items, such as rating items from one questionnaire.
2. Set the number of factors
   Enter how many factors to extract. It must be smaller than the number of items. The starting value is 2.
3. Choose the rotation
   Varimax (orthogonal), Oblimin (oblique) or None (unrotated).
4. Choose the extraction method
   Principal Axis Factoring (PAF) or Principal Components (PC-based).
5. Apply a subsample filter if needed
   Use it to run the analysis for one group, for example to see whether the structure holds for women and men separately.
6. Click Run EFA
   Check the Bartlett banner and KMO first, then read the loadings, the variance, the communalities and, for Oblimin, the factor correlations. Export the tables as CSV, Excel or Word, or add the result to AnalyZense or to a Data Canvas story.

People with a missing value on any selected item are left out, and the number of cases used is shown with the title of the results. Before you run the analysis, look at how each item is distributed and check the data for gaps and errors.

07Interpreting the result

The figures below are invented to show the steps, and they follow the results tables in AnalyZ Solutions, with a few columns left out to save space. They come from the health worker survey described earlier: 300 people, ten items rated from 1 to 5, three factors, principal axis factoring and Varimax rotation.

Bartlett's test gives chi-square(45) = 742.5, p < 0.001, and the KMO value is 0.769, labelled middling.

Reading it step by step

1. Check the suitability
   Bartlett's test is significant and KMO is 0.769. The items are correlated enough for a factor analysis.
2. Decide on the number of factors
   Three eigenvalues are above 1 (3.09, 1.88 and 1.31), and the fourth drops to 0.71. That agrees with the three ideas the questionnaire was written around, so a three-factor solution is a sensible choice.
3. Read the loadings
   Factor 1 collects the four items about meaning (loadings 0.46 to 0.78). Factor 3 collects the three supervision items (0.66 to 0.69). Factor 2 collects the three workload items (0.63 to 0.67). Every item loads at 0.40 or more on its own factor and below 0.30 on the others, so there are no cross-loaders. The factors can be named Meaning in work, Supervision and Workload.
4. Look at the communalities
   Nine items are between 0.41 and 0.65. I plan to stay here has 0.286, below 0.30, and is highlighted. Its loading on Factor 1 is 0.46, the weakest of the four. Intention to stay probably depends on much more than how meaningful the work feels, such as pay and family circumstances.
5. Look at the variance
   The three factors explain 47.1% of the variation in the items, which the module calls moderate. The rest is variation specific to each item and measurement error. A figure of this size is common for short rating scales.
6. Check each group with alpha
   Alpha is 0.75 for the four meaning items, 0.75 for supervision and 0.69 for workload. Workload falls in the questionable band, and has only three items. Without I plan to stay here, the three meaning items give 0.77, so that item may be worth dropping or analysing on its own.

In plain words

A factor analysis of ten items from 300 health workers found three factors, which together explain 47% of the variation in the answers: meaning in work, supervision and workload. All items loaded clearly on the factor they were written for. One item, I plan to stay here, was only weakly explained by the factors. This analysis describes the structure in this sample. A confirmatory analysis in a new sample would be needed to confirm it.

08Common mistakes

Treating the result as proof. EFA is exploratory. A structure found in one sample should be confirmed in another.

Choosing the number of factors by the Kaiser rule alone. Look at the scree plot, compare solutions and consider the meaning of each factor.

Using Varimax when the factors are clearly related. The forced independence distorts the loadings. Check the factor correlations from an Oblimin run.

Dropping many items at once. Each removal changes the loadings of the others. Remove one at a time and rerun.

Ignoring cross-loaders and low communalities. They signal items that are ambiguous or poorly explained, and they weaken the subscales built from them.

Forgetting wording effects. Negatively worded items sometimes form a factor of their own that reflects the wording and not the content. Check how reverse-worded items behave.

Naming factors too early. Read the content of every item on a factor before you choose a label.

Too little data. Small samples give unstable loadings. Aim for at least five respondents per item, and more when you can.