The Synthetic Imputation Approach
Using LLMs to Generate Synthetic Texts for Underrepresented Categories in Supervised Classification Tasks
Bibliographic Data
| ID | 24228934 |
|---|---|
| Authors | Joan C Timoneda (0000-0001-7057-872X, Purdue University) |
| Year | 2026 |
| Pages | 1-15 |
| Publication date | 2026-10-14 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Political Analysis (JOURNAL) |
| Journal identifiers | ISSN: 1047-1987 • E-ISSN: 1476-4989 |
| Publisher | Cambridge University Press (CUP) (PUBLISHER) |
| DOI | 10.1017/pan.2026.10057 |
| OpenAlex | W7221813547 |
| Language | EN |
| References cited | 29 |
Annotated text data are central to many empirical applications in political science, yet many annotation tasks involve categories that are sparsely populated. These low- N categories pose problems for both encoder-only and decoder-only (generative) Transformer large language models (LLMs), leading to low performance, biased evaluation and measurement error in downstream analyses. This article introduces the synthetic imputation approach, a targeted method for expanding sparse categories while remaining consistent with the category-defining content of the human-coded examples. The approach uses a generative LLM to produce additional texts for underrepresented categories by conditioning on small, rotating subsets of human-coded examples. Rather than paraphrasing or replacing the original data, synthetic imputation adds informative variation only where data are thin. I evaluate the approach using both encoder-only and decoder-only LLM classifiers across multiple political science applications. The results show that synthetic imputation substantially improves performance and evaluation stability in sparse settings, recovering full-sample performance once a modest number of true observations is available, with limited and predictable overfitting at very low levels of N .
evaluation stability · large language models · Overfitting · political science data · supervised classification · synthetic text generation · Computational and Text Analysis Methods · Ethics and Social Impacts of AI · Topic Modeling
Bart
ChatGPT outperforms crowd workers for text-annotation tasks
Estimating group fixed effects in panel data with a binary dependent variable
Nostalgia in European Party Politics
Bert, RoBerta, or DeBerta? Comparing Performance Across Transformers Models in Political Science Text
How to train your stochastic parrot
Codebook LLMs
Synthetically generated text for supervised text analysis
Text as Data
Computer-Assisted Text Analysis for Comparative Politics
Causal Inference with Latent Treatments
| Highly cited | No |
|---|