Updating 'The Future of Coding
Qualitative Coding with Generative Large Language Models
Bibliographic Data
| ID | 2331211 |
|---|---|
| Authors | Nga Than (0000-0002-6845-6253, Stone Center on Socio-Economic Inequality, The Graduate Center, CUNY, New York City, USA), Leanne Fan (Stone Center on Socio-Economic Inequality, The Graduate Center, CUNY, New York City, USA), Tina Law (0000-0001-7631-6763, University of California, Davis), Livingston Krumah Nelson (0000-0001-8948-300X, University of British Columbia, corresponding author), Leslie Mccall (0000-0002-7700-3969, Stone Center on Socio-Economic Inequality, The Graduate Center, CUNY, New York City, USA) |
| Year | 2025 |
| Volume | 54 |
| Issue | 3 |
| Pages | 849-888 |
| Publication date | 2025-08-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Sociological Methods & Research (JOURNAL) |
| Journal identifiers | ISSN: 0049-1241 • E-ISSN: 1552-8294 |
| Publisher | SAGE Publications Inc (PUBLISHER) |
| DOI | 10.1177/00491241251339188 |
| OpenAlex | W4410554739 |
| Language | EN |
| Citations received | 13 |
| References cited | 40 |
Over the past decade, social scientists have adapted computational methods for qualitative text analysis, with the hope that they can match the accuracy and reliability of hand coding. The emergence of GPT and open-source generative large language models (LLMs) has transformed this process by shifting from programming to engaging with models using natural language, potentially mimicking the in-depth, inductive, and/or iterative process of qualitative analysis. We test the ability of generative LLMs to replicate and augment traditional qualitative coding, experimenting with multiple prompt structures across four closed- and open-source generative LLMs and proposing a workflow for conducting qualitative coding with generative LLMs. We find that LLMs can perform nearly as well as prior supervised machine learning models in accurately matching hand-coding output. Moreover, using generative LLMs as a natural language interlocutor closely replicates traditional qualitative methods, indicating their potential to transform the qualitative research process, despite ongoing challenges
Coding (social sciences) · Generative grammar · Linguistics · Natural language processing · Qualitative research · Social science · Sociology · Artificial Intelligence · Computational and Text Analysis Methods · Computer Science · Natural Language Processing Techniques · Topic Modeling
Generative Multimodal Models for Social Science
Meeting the machines half-way
Running With Scissors? Integrating GPT Models Into Public Policy Research
Introducing Halc
Should Generative AI be Used in Reflexive Qualitative Research
Religion and Racial Bias in Artificial Intelligence Large Language Models
One Sentiment, Multiple Interpretations
Who believes in science? A computational tool for identifying language invoking or disputing scientific knowledge
Thinking the unthinkable
Administrative Decision-Making with Generative AI
Computational Text Analysis for Building and Testing Social Theory
Multi-Agent Ethnography
Integrating Generative Artificial Intelligence into Social Science Research
Content analysis
Why Americans Hate Welfare
Shaping Abortion Discourse
ChatGPT outperforms crowd workers for text-annotation tasks
Can Generative AI improve social science?
Politics as Usual? Measuring Populism, Nationalism, and Authoritarianism in U.S. Presidential Campaigns (1952–2020) with Neural Language Models
The Undeserving Rich
The Extended Computational Case Method
Abductive Logic of Inquiry for Quantitative Research in the Digital Age
A Methodological Framework for the Sociology of Culture
Open-source LLMs for text annotation
The Meaning of Action
Language Models in Sociological Research
Training Computational Social Science PhD Students for Academic and Non-Academic Careers
Friction by Machine
Can Large Language Models Transform Computational Social Science
Ontologies, methodologies, and new uses of Big Data in the social and cultural sciences
Measuring nominal scale agreement among many raters
Text as Data
Start Generating
Intercoder Reliability in Qualitative Research
Machine Bias. How Do Generative Language Models Answer Opinion Polls
Large Language Models for Text Classification
The Augmented Social Scientist
From Strange to Normal
When Corporations Are People
Flexible Coding of In-depth Interviews
The Future of Coding
Computational Grounded Theory
Algorithmic Abduction
Cycles of Conflict, a Century of Continuity
| Unique citing works | 13 |
|---|---|
| Citations per year | 13 |
| Citation span | 2025 - 2026 (2) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 12 |