Survey Quality and Acquiescence Bias
A Cautionary Tale
Bibliographic Data
| ID | 12370752 |
|---|---|
| Authors | Andres Cristobal Cruz (0000-0003-3819-9433, The University of Texas at Austin), Andrés Cruz (The University of Texas at Austin), Adam Bouyamourn (0000-0002-2073-2132, Princeton University), Joseph T Ornstein (0000-0002-5704-2098, University of Georgia, corresponding author) |
| Year | 2026 |
| Volume | 34 |
| Issue | 3 |
| Pages | 1-8 |
| Publication date | 2026-01-12 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Political Analysis (JOURNAL) |
| Journal identifiers | ISSN: 1047-1987 • E-ISSN: 1476-4989 |
| Publisher | Cambridge University Press (CUP) (PUBLISHER) |
| DOI | 10.1017/pan.2025.10030 |
| OpenAlex | W7122644768 |
| Language | EN |
| References cited | 16 |
In this note, we offer a cautionary tale on the dangers of drawing inferences from low-quality online survey datasets. We reanalyze and replicate a survey experiment studying the effect of acquiescence bias on estimates of conspiratorial beliefs and political misinformation. Correcting a minor data coding error yields a puzzling result: respondents with a postgraduate education appear to be the most prone to acquiescence bias. We conduct two preregistered replication studies to better understand this finding. In our first replication, conducted using the same survey platform as the original study, we find a nearly identical set of results. But in our second replication, conducted with a larger and higher-quality survey panel, this apparent effect disappears. We conclude that the observed relationship was an artifact of inattentive and fraudulent responses in the original survey panel, and that attention checks alone do not fully resolve the problem. This demonstrates how “survey trolls” and inattentive respondents on low-quality survey platforms can generate spurious and theoretically confusing results
Acquiescence · Artifact (error · Data quality · Quality (philosophy · Replicate · Set (abstract data type · Spurious relationship · Survey data collection · Survey research · Mobile Crowdsensing and Crowdsourcing · Survey Methodology and Nonresponse · Survey Sampling and Estimation Techniques
Current research overstates American support for political violence
Fraud in Online Surveys
Education Moderates Some Response Effects in Attitude Measurement
Don’t Know” Responses, Personality, and the Measurement of Political Knowledge
A Text-As-Data Approach for Using Open-Ended Responses as Manipulation Checks
An Improved Question Format for Measuring Conspiracy Beliefs
Analyze the attentive and bypass bias
Acquiescence Bias Inflates Estimates of Conspiratorial Beliefs and Political Misperceptions
Acquiescence Bias Inflates Estimates of Conspiratorial Beliefs and Political Misperceptions - Corrigendum
Reconsidering the Measurement of Political Knowledge
Separating the Shirkers from the Workers? Making Sure Respondents Pay Attention on Self‐Administered Surveys
Measuring Misperceptions
Social Media and Fake News in the 2016 Election
| Citation velocity | historical |
|---|---|
| Highly cited | No |