Detecting pro-kremlin disinformation using large language models

Bibliographic Data

ID6448535
AuthorsMarianne Kramer (University of Copenhagen), Yevgeniy Golovchenko (0000-0003-3292-7372, University of Copenhagen), Frederik Hjorth (0000-0003-4063-4983, University of Copenhagen)
Year2025
Volume12
Issue2
Publication date2025-04-01
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueResearch & Politics (JOURNAL)
Journal identifiersISSN: 2053-1680 • E-ISSN: 2053-1680
PublisherSAGE Publishing (PUBLISHER • US)
DOI10.1177/20531680251351910
OpenAlexW4411580328
LanguageEN
References cited15

A growing body of literature examines manipulative information by detecting political mis-/disinformation in text data. This line of research typically involves highly costly manual annotation of text for manual content analysis, and/or training and validating automated downstream approaches. We examine whether Large Language Models (LLMs) can detect pro-Kremlin disinformation about the war in Ukraine, focusing on the case of the downing of the civilian flight MH17. We benchmark methods using a large set of tweets labeled by expert annotators. We show that both open and closed LLMs can accurately detect pro-Kremlin disinformation tweets, outperforming both a research assistant and supervised models used in earlier research and at drastically lower cost compared to either research assistants or crowd workers. Our findings contribute to the literature on mis/-disinformation by showcasing how LLMs can substantially lower the costs of detection even when the labeling requires complex, context-specific knowledge about a given case

Disinformation · Fake news · Internet privacy · Political science · Social media · World Wide Web · Computer Science · Cybersecurity and Cyber Warfare Studies · Intelligence, Security, War Strategy · Misinformation and Its Impacts

Tools

Open DOIOpen Access