How Model Choice and Memory Shape Preference Consistency in Large Language Models
Bibliographic Data
| ID | 24212965 |
|---|---|
| Authors | Yick Chung (0009-0007-9286-279X, University of Milan) |
| Year | 2026 |
| Publication date | 2026-09-29 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Sociological Methods & Research (JOURNAL) |
| Journal identifiers | ISSN: 0049-1241 • E-ISSN: 1552-8294 |
| Publisher | SAGE Publications (PUBLISHER • US) |
| DOI | 10.1177/00491241261492203 |
| OpenAlex | W7214744109 |
| Language | EN |
| References cited | 36 |
When a language model answers a sequence of questions, researchers report the model and temperature but rarely what or how it was told about its own earlier answers. I show that this underreported setting behaves as a mode effect. Testing four large language model (LLM) families with revealed preference theory (GARP) across moral, economic, and social tasks ( N = 5,926 ), I find that how prior choices re-enter the prompt, the memory condition , moves the share of internally consistent subjects by up to 48 percentage points, mostly where that share starts lowest. Memory also influences which preferences the models express and, in a silicon-sampling extension, it brings the answers closer to the human distribution. Nonetheless, gains in consistency and surface fidelity do not necessarily translate into representativeness: a consistent, human-looking model may not represent any human population. For LLMs as social-science instruments, this study offers promise, caution, and a method for testing the stability of sequential outputs.
economic preferences · GARP · large language models · memory effects · Model selection · moral preferences · preference consistency · prompt engineering · representativeness · Revealed preference · Social preferences · Computational and Text Analysis Methods · Ethics and Social Impacts of AI · Topic Modeling
The Construction of Utility Functions from Expenditure Data
Durably reducing conspiracy beliefs through dialogues with AI
The Nonparametric Approach to Demand Analysis
A weakly informative default prior distribution for logistic and other regression models
Generative Agents
Giving According to Garp
ChatGPT outperforms crowd workers for text-annotation tasks
The weirdest people in the world?
Internet, Phone, Mail, and Mixed‐Mode Surveys
A Note on the Pure Theory of Consumer's Behaviour
Out of One, Many
Self-reports
Can Large Language Models Transform Computational Social Science
Playing repeated games with large language models
Balancing Large Language Model Alignment and Algorithmic Fidelity in Social Science Research
Integrating Generative Artificial Intelligence into Social Science Research
Simulating Subjects
Machine Bias. How Do Generative Language Models Answer Opinion Polls
| Citation velocity | historical |
|---|---|
| Highly cited | No |