Using large language models to analyze political texts through natural language understanding

Bibliographic Data

ID11065069
AuthorsKenneth Benoit (0000-0002-0797-564X, Open Data Institute, corresponding author), Scott De Marchi (0000-0003-4355-9800, Duke University), Conor Laver (Northeastern University), Michael Laver (0000-0001-7515-9831, Open Data Institute), Jinshuai Ma (0009-0009-2040-249X, Open Data Institute)
Year2026
Publication date2026-03-19
Open AccessYes
TypeARTICLE
VenueAmerican Journal of Political Science (JOURNAL)
Journal identifiersISSN: 0092-5853 • E-ISSN: 1540-5907
PublisherWiley (PUBLISHER • GB)
DOI10.1111/ajps.70050
OpenAlexW7138869444
LanguageEN
Citations received4
References cited34

Large language models (LLMs) offer scalable alternatives to human experts when analyzing political texts for meaning , using natural language understanding (NLU). Qualitative NLU methods relying on human experts are severely limited by cost and scalability. Statistical text‐as‐data methods are scalable but rely on strong and often unrealistic assumptions. We propose a systematic, scalable, and replicable method that can extend existing qualitative and quantitative approaches by using LLMs to interpret texts meaningfully rather than as mere data. Our ensemble means of LLM‐generated estimates of party positions on six key issue dimensions correlate highly with equivalent mean ratings by country specialists. When applied to coalition policy declarations, LLM estimates align more closely with standard models of government formation than hand‐coded estimates. We conclude with a discussion of the profound implications of modern LLMs for political text analysis

Government (linguistics · Key (lock · Language model · Natural (archaeology · Natural language · Natural language understanding · Politics · Scalability · Computational and Text Analysis Methods · Populism, Right-Wing Movements · Sentiment Analysis and Opinion Mining

Tools

Open DOIOpen Access