Scaling Up With Integrity
Valid and Efficient Narrative Policy Framework Analyses Using Large Language Models
Bibliographic Data
| ID | 6330840 |
|---|---|
| Authors | Kylie L Anglin (0000-0001-7661-3370, Neag School of Education University of Connecticut Storrs Connecticut USA, corresponding author), Ariell Rose Bertrand (0000-0002-3974-0642, College of Education Michigan State University East Lansing Michigan USA), Julie Gottlieb (0000-0002-1204-8496, College of Education Texas Tech University Lubbock Texas USA), Jessica J Gottlieb (0000-0001-7576-8047, Texas Tech University), Jessica Gottlieb, Joseph Elefante (0000-0003-4683-3094, College of Education Texas Tech University Lubbock Texas USA) |
| Year | 2025 |
| Publication date | 2025-06-09 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Policy Studies Journal (JOURNAL) |
| Journal identifiers | ISSN: 0190-292X • E-ISSN: 1541-0072 |
| Publisher | Wiley (PUBLISHER • GB) |
| DOI | 10.1111/psj.70045 |
| OpenAlex | W4411159523 |
| Language | EN |
| Citations received | 4 |
| References cited | 28 |
Given vast quantities of digital and online data—such as news articles, congressional testimony, and social media posts—the potential for large scale narrative analyses has dramatically increased. Narrative Policy Framework (NPF) researchers can now access thousands or even millions of policy‐relevant documents. However, these large datasets also bring new challenges, including how to scale analyses while maintaining validity and reliability. Traditionally, NPF studies employ labor intensive human coding to identify narrative elements, limiting the number of documents that can be feasibly analyzed. Recognizing the potential of large language models (LLMs) to automate such analyses, this study demonstrates the use of zero‐shot, few‐shot, and fine‐tuned classification techniques to identify narrative elements. By prioritizing construct validity at each step in the process, we offer a rigorous and replicable approach to integrating LLMs into narrative policy research
Linguistics · Narrative · Narrative review · Political science · Scaling · Computational and Text Analysis Methods · Computer Science · Mathematics · Philosophy · Psychology · Topic Modeling
Applied Logistic Regression
ChatGPT
Demystifying Content Analysis
Interrater reliability
Practical and ethical challenges of large language models in education
Persuasion with Precision
Trump's tweets as policy narratives
Talking Policy in Congressional Campaigns
Fracking Twitter
Strategically constructed narratives on artificial intelligence
Advocacy Group Messaging on Social Media
Narrative Strategies in the Gun Policy Debate
Text as Data
Relatio
Constructing Policy Narratives in 140 Characters or Less
A Narrative Policy Framework
The devil we know and the angel that did not fly
Holding out the promise of Lasswell's dream
| Citations per year | 4 |
|---|---|
| Citation span | 2026 - 2026 (1) |
| Highly cited | No |