Disentangling Item and Testing Effects in Inoculation Research on Online Misinformation

Solomon Revisited

Bibliographic Data

ID20284099
AuthorsJon Roozenbeek (0000-0002-8150-9305, University of Cambridge, Cambridge, UK, corresponding author), Rakoen Maertens (0000-0001-8507-5359, University of Cambridge, Cambridge, UK), William P Mcclanahan (0000-0002-6604-3842, University of Cambridge), William McClanahan (University of Cambridge, Cambridge, UK), Sander Van Der Linden (0000-0002-0269-1744, University of Cambridge, Cambridge, UK)
Year2021
Volume81
Issue2
Pages340-362
Publication date2021-04-01
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueEducational and Psychological Measurement (JOURNAL)
Journal identifiersISSN: 0013-1644 • E-ISSN: 1552-3888
PublisherSAGE Publications (PUBLISHER • US)
DOI10.1177/0013164420940378
PMID37929261
OpenAlexW4239239283
LanguageEN
Citations received26
References cited30

Online misinformation is a pervasive global problem. In response, psychologists have recently explored the theory of psychological inoculation: If people are preemptively exposed to a weakened version of a misinformation technique, they can build up cognitive resistance. This study addresses two unanswered methodological questions about a widely adopted online “fake news” inoculation game, Bad News. First, research in this area has often looked at pre- and post-intervention difference scores for the same items, which may imply that any observed effects are specific to the survey items themselves (item effects). Second, it is possible that using a pretest influences the outcome variable of interest, or that the pretest may interact with the intervention (testing effects). We investigate both item and testing effects in two online studies (total N = 2,159) using the Bad News game. For the item effect, we examine if inoculation effects are still observed when different items are used in the pre- and posttest. To examine the testing effect, we use a Solomon’s Three Group Design. We find that inoculation interventions are somewhat influenced by item effects, and not by testing effects. We show that inoculation interventions are effective at improving people’s ability to spot misinformation techniques and that the Bad News game does not make people more skeptical of real news. We discuss the larger relevance of these findings for evaluating real-world psychological interventions

Cognitive psychology · Intervention (counseling) · Misinformation · Psychological intervention · Skepticism · Test (biology) · Applied Psychology · Computer Science · Media Influence and Politics · Misinformation and Its Impacts · Psychology · Social Media and Politics · Social Psychology