Catch Questions Can Increase Lookups on Political Knowledge Questions

Bibliographic Data

ID6373929
AuthorsMatthew H Graham (0000-0003-4808-2984, Temple University)
Year2025
Volume89
Issue2
Pages415-423
Publication date2025-08-01
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenuePublic Opinion Quarterly (JOURNAL)
Journal identifiersISSN: 0033-362X • E-ISSN: 1537-5331
PublisherOxford University Press (OUP) (PUBLISHER)
DOI10.1093/poq/nfaf015
OpenAlexW4410794470
LanguageEN
Citations received1
References cited14

Surveys commonly use “catch” or “trap” questions to identify subjects who are likely to look up the answers to knowledge questions. This note shows that catch questions can backfire, causing an increase in the proportion of respondents who look up the answers to subsequent questions. In two preregistered survey experiments in online convenience samples, a catch question was randomly assigned to appear before or after a knowledge battery. When the catch question appeared first, window switching on the knowledge questions was about 25 percent more common. This suggests that when they are used, catch questions should be placed at the end of knowledge batteries, and should also not precede later items that could be affected by lookups. More generally, the results highlight the importance of pre-testing catch and trap questions for unintended consequences

Political science · Politics · Educator Training and Historical Pedagogy · Evaluation and Performance Assessment · Innovative Teaching Methodologies in Social Sciences · Law

Tools

Open DOI