Experimental comparison of a chatbot-like questionnaire design with a traditional web questionnaire design

Bibliographic Data

ID12102438
AuthorsCeyda Çavuşoğlu Deveci (corresponding author), Marek Fuchs, Anke Metzler (Technical University of Darmstadt, Darmstadt; Germany)
Year2026
Volume169-170
Issue1
Pages213-235
Publication date2026-03-06
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueBulletin of Sociological Methodology/Bulletin de Méthodologie Sociologique (JOURNAL)
Journal identifiersISSN: 0759-1063 • E-ISSN: 2070-2779
PublisherSAGE Publishing (PUBLISHER • US)
DOI10.1177/07591063261424249
OpenAlexW7135055918
LanguageEN
References cited47

Web surveys have become the dominant mode of survey data collection. They offer advantages in terms of cost and time over other modes and they are the new normal in survey research for many populations and in many fields of study. Nevertheless, web surveys, like other survey modes, are affected by the consequences of satisficing behavior, which is attributed, among other things, to low motivation among respondents. We assume that the motivation of respondents can be increased by using a respondent-friendly approach in the design of the questionnaire ( Dillman, 2000 ), which was achieved by implementing a chatbot-like questionnaire interface. In a randomized field-experiment conducted among university students we employed a between-subject design comparing a chatbot-like interface and a traditional web survey design administering the same questionnaire. We assessed respondent evaluation, as well as data quality indicators like response time, non-differentiation, item missing rates, and the length of answers to narrative open-ended questions. Results indicate that respondents perceived the chatbot-like design as more original and entertaining than the traditional web survey design. By contrast, participants rated the chatbot-like interface as more difficult to navigate. The analyses of response time and character count of answers to open-ended questions showed no significant differences between the two designs. The proportion of respondents with at least one item-missing was marginally lower in the chatbot-like design, while the degree of differentiation of one of the multi-item scales was higher in the web survey design

Computer-assisted web interviewing · Quality (philosophy · Questionnaire · Respondent · Satisficing · Survey data collection · Survey research · Web survey · AI in Service Interactions · Mobile Crowdsensing and Crowdsourcing · Survey Methodology and Nonresponse

Tools

Open DOI