Question order bias revisited
A split‐ballot experiment on satisfaction with public services among experienced and professional users
Bibliographic Data
| ID | 10388668 |
|---|---|
| Authors | R J Lawrence (0000-0001-9412-7296, VIVE—The Danish Center for Social Science Research Aarhus and Copenhagen Denmark, corresponding author), Maria Falk Mikkelsen (0000-0001-6256-6403, VIVE—The Danish Center for Social Science Research Aarhus and Copenhagen Denmark), Morten Hjortskov (0000-0001-6631-7017, Department of Political Science Aarhus University Aarhus Denmark), Mogens Jin Pedersen (0000-0001-9428-3395, Department of Political Science University of Copenhagen Copenhagen Denmark) |
| Year | 2020 |
| Volume | 99 |
| Issue | 1 |
| Pages | 189-204 |
| Publication date | 2020-06-28 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Public Administration (JOURNAL) |
| Journal identifiers | ISSN: 0033-3298 • E-ISSN: 1467-9299 |
| Publisher | Wiley (PUBLISHER • GB) |
| DOI | 10.1111/padm.12688 |
| OpenAlex | W3037866210 |
| Language | EN |
| Citations received | 16 |
| References cited | 41 |
Public decision‐makers increasingly rely on satisfaction surveys to inform budget and policy decisions. Yet, our knowledge of whether, and under what conditions, this input from public service users provides valid performance information remains incomplete. Using a preregistered split‐ballot experiment among government grant recipients in Denmark, this article shows that the ordering of survey questions can bias satisfaction measures even for highly experienced and professional respondents. We find that asking about overall satisfaction before any specific service ratings lowers overall user satisfaction, compared to the reverse order, while the correlations between specific ratings and overall satisfaction are relatively stable. Also, the question order effect outweighs that of a large‐scale embezzlement scandal, which unexpectedly hit the investigated government agency during the data collection. Our results support rising concerns that subjective performance indicators are susceptible to bias. We discuss how practitioners should approach satisfaction surveys to account for the risk of question order bias
Ballot · Business · Order (exchange · Political science · Public relations · Voting · Electoral Systems and Political Participation · Finance · Law · Patient Satisfaction in Healthcare · Psychology · Social Psychology · Survey Methodology and Nonresponse
Does Order Matter? The Impact of Methodological Decisions in Survey Research Concurrently Examining Attachment to Mothers and Fathers
Revisiting an “urban legend”
Testing the generalizability of the white leadership standard in the post-Obama era
Abrupt peaks in perceived risk of occasional drug use after changing the question order in a repeated self-administered survey
Reporting multiple dimensions of public service performance
Local valuation of ecosystem services from protected areas along a forest–savanna gradient in West Africa
Activating employees’ motivation to increase intentions to report wrongdoings
Chronic Stress Is Associated with Reduced Mindful Acceptance Skills but Not with Mindful Attention Monitoring
Willingness of population health survey participants to provide personal health information and biological samples
Towards a Valid Measure of the Economic Value of Museum Experiences
The association between food neophobia, bi-dimensional aspects of orthorexia, and anxiety among vegetarians and omnivores
Question-Order Effect in the Study of Satisfaction with Democracy
Compliance burden versus program integrity protection
The consistency of question‐order bias in a changing political context
Testing order effects in autobiographical memory research
The Effect of Performance Failures on User Satisfaction
The Nature and Origins of Mass Opinion
Attitude Construction
Mental Construal and the Emergence of Assimilation and Contrast Effects
The Psychology of Survey Response
Expectations, performance, and citizen satisfaction with urban services
An Experimental Test of the Expectancy-Disconfirmation Theory of Citizen Satisfaction
Citizen Expectations and Satisfaction Over Time
The effects of question order and response-choice on self-rated health status in the English Longitudinal Study of Ageing (Elsa)
Objective and Subjective Performance Measures
Citizen (Dis)satisfaction
Priming and context effects in citizen satisfaction surveys
The Order of Questions in a Survey on Citizen Satisfaction With Public Services
Attitude accessibility as a determinant of object construal and evaluation
Education Moderates Some Response Effects in Attitude Measurement
Effects of Question Order on Survey Responses
Measuring Satisfaction With Organizations
Assimilation and Contrast Effects in Part-Whole Question Sequences
Question-Order Effects on Presidential Popularity
A Question Order Effect in a Local Government Survey
The Dilemma of the Unsatisfied Customer in a Market Model of Public Administration
Citizen Satisfaction with Urban Services
Evaluating Urban Public Schools
RNICE Model
Measuring Street Cleanliness
Citizen Satisfaction and the Kaleidoscope of Government Performance
Cognitive Biases in Performance Evaluations
A Conceptual Framework for Analysis of Goal Ambiguity in Public Organizations
Best Practice Recommendations for Replicating Experiments in Public Administration
Citizens’ Evaluations of the Public Sector
Unconscious Bias in Citizens’ Evaluations of Public Sector Performance
Motivated Reasoning about Public Performance
Evaluating the Expectations Disconfirmation and Expectations Anchoring Approaches to Citizen Satisfaction with Local Public Services
The Political Psychology of Citizen Satisfaction
Encouraging Political Voices of Underrepresented Citizens through Coproduction
Generalization of Classic Question Order Effects Across Cultures
| Unique citing works | 16 |
|---|---|
| Citations per year | 4 |
| Citation span | 2022 - 2026 (5) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 15 |