Nonresponse Bias: Definition, Causes, and Prevention

What is nonresponse bias?

Nonresponse bias is a form of sampling bias that occurs when the characteristics of non-respondents systematically differ from those of respondents on variables relevant to the study outcome. It is distinct from mere nonresponse error (a reduction in precision from having a smaller sample): nonresponse bias is a directional distortion that shifts estimates away from the true population value.

The degree of bias depends on two factors: (1) the rate of nonresponse, and (2) the difference between respondents and non-respondents on the outcome of interest. A low response rate does not guarantee large bias if the two groups are similar; conversely, even a modest nonresponse rate can produce severe bias if non-respondents are very different from respondents.

Statisticians formalize this with the expression: Bias = (nonresponse rate) × (mean difference between respondents and non-respondents). Both terms must be considered when evaluating the potential impact of nonresponse.

Causes of nonresponse bias

Survey fatigue

Individuals who are contacted frequently for surveys — particularly in panel studies — develop fatigue and begin to decline or respond carelessly. Because frequent survey participants tend to be more engaged with a topic, their withdrawal leaves a residual sample that is less representative.

Topic salience

People with strong opinions or direct experience with the survey topic are more likely to respond than those who are indifferent. A survey on chronic pain management, for instance, will disproportionately attract respondents who experience chronic pain, inflating estimated prevalence in the general population.

Sensitive or stigmatized content

Questionnaires covering mental health, drug use, income, or sexual behavior experience higher refusal rates among the subpopulations most relevant to the study — precisely those whose data are most needed. The resulting sample underestimates the true prevalence of the sensitive attribute.

Accessibility and literacy barriers

Online surveys exclude populations with limited internet access; written questionnaires exclude those with low literacy. In both cases, socioeconomic and demographic characteristics are confounded with response, producing systematic nonresponse bias.

Differential follow-up in longitudinal studies

Participants who move frequently, have unstable housing, or experience adverse health events are harder to re-contact in longitudinal research. Because these characteristics often correlate with study outcomes, their loss to follow-up biases estimates of change over time.

Nonresponse bias in research

Nonresponse bias is particularly consequential in three research contexts:

National surveys and polling

Telephone and online polls have seen response rates decline from roughly 35% in the 1990s to single digits in many contemporary studies (Pew Research Center, 2019). Whether these low rates produce large biases depends on whether the declining respondents differ from remaining ones on political, demographic, or attitudinal variables. Evidence suggests that engagement, trust in institutions, and political interest are all higher among survey respondents than the general public, introducing systematic slants in political polling.

Clinical and epidemiological research

In cohort studies, loss to follow-up — a form of nonresponse — is frequently correlated with outcomes. Patients who miss follow-up appointments may have had worse outcomes, died, or recovered fully (and no longer felt the need to return). Both directions can bias survival estimates and incidence rates.

Educational research

School-based surveys often permit students or parents to opt out. Students whose parents refuse consent tend to differ from consenting students on socioeconomic status, academic achievement, and behavioral indicators — all of which are commonly studied outcomes.

Examples of nonresponse bias

Workplace satisfaction surveys

Employees who are disengaged or planning to resign often skip internal satisfaction surveys, leaving results that overstate organizational morale. HR teams acting on these inflated results may miss early warning signs of turnover.

Health behavior studies

Studies of exercise habits that recruit through gym posters or fitness apps disproportionately enroll active individuals. People who are sedentary — and therefore most relevant to a study on barriers to exercise — are systematically absent from the sample.

Vaccine safety surveillance

Passive adverse event reporting systems rely on self-initiated reports. Individuals who experience no side effects have no reason to report; severe events may prevent reporting entirely. The resulting data overrepresent moderate adverse events relative to no-event and severe-event outcomes.

How to prevent nonresponse bias

Maximize response rates through design

Shorter surveys, clear and relevant language, mobile-friendly formats, and advance notification all reduce the burden of participation and increase response rates. Total Survey Error frameworks recommend treating nonresponse as a quality dimension from the initial design phase, not as an afterthought.

Use multiple contact modes

Combining mail, phone, email, and in-person follow-up reduces the likelihood that any single accessibility barrier excludes a subpopulation. Dillman's Tailored Design Method recommends at least four contact attempts across modes to maximize coverage.

Offer incentives carefully

Modest unconditional incentives (a small prepaid gift) increase response rates without creating the self-selection problem associated with large conditional rewards. Conditional incentives (pay only those who complete) attract motivated responders and worsen self-selection bias.

Conduct nonresponse follow-up

A random subsample of non-respondents should be intensively contacted to collect at least a short set of key variables. Comparing respondent and non-respondent answers on these variables quantifies the bias and can inform post-hoc weighting adjustments.

Apply post-stratification weighting

When auxiliary population data exist (e.g., from a census), survey weights can adjust the sample to match known population distributions on age, sex, education, and geography. While weighting reduces nonresponse bias on measured variables, it cannot correct for unmeasured differences between respondents and non-respondents.

Use multiple imputation for missing data

When nonresponse is partly at the item level (some participants skip individual questions), multiple imputation using other observed variables can recover unbiased estimates under the assumption that data are missing at random (MAR). Sensitivity analyses under more pessimistic assumptions (missing not at random, MNAR) should accompany any imputation strategy.

Quick summary