Open-ended versus Closed Probes
Assessing Different Formats of Web Probing
Bibliographic Data
| ID | 2331069 |
|---|---|
| Authors | Cornelia E Neuert (0000-0001-9855-5618, GESIS—Leibniz Institute for the Social Sciences, Mannheim, P.O. Box 12 21 55, 68072 Mannheim, Germany, corresponding author), Katharina Meitinger (0000-0001-8160-556X, Utrecht University, the Netherlands), Dorothée Behr (0000-0002-2653-0531, GESIS—Leibniz Institute for the Social Sciences, Mannheim, P.O. Box 12 21 55, 68072 Mannheim, Germany) |
| Year | 2023 |
| Volume | 52 |
| Issue | 4 |
| Pages | 1981-2015 |
| Publication date | 2023-11-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Sociological Methods & Research (JOURNAL) |
| Journal identifiers | ISSN: 0049-1241 • E-ISSN: 1552-8294 |
| Publisher | SAGE Publications Inc (PUBLISHER) |
| DOI | 10.1177/00491241211031271 |
| OpenAlex | W3184916581 |
| Language | EN |
| Citations received | 12 |
| References cited | 31 |
The method of web probing integrates cognitive interviewing techniques into web surveys and is increasingly used to evaluate survey questions. In a usual web probing scenario, probes are administered immediately after the question to be tested (concurrent probing), typically as open-ended questions. A second possibility of administering probes is in a closed format, whereby the response categories for the closed probes are developed during previously conducted qualitative cognitive interviews. Using closed probes has several benefits, such as reduced costs and time efficiency, because this method does not require manual coding of open-ended responses. In this article, we investigate whether the insights gained into item functioning when implementing closed probes are comparable to the insights gained when asking open-ended probes and whether closed probes are equally suitable to capture the cognitive processes for which traditionally open-ended probes are intended. The findings reveal statistically significant differences with regard to the variety of themes, the patterns of interpretation, the number of themes per respondent, and nonresponse. No differences in number of themes across formats by sex and educational level were found
Coding (social sciences) · Cognition · Cognitive interview · Cognitive psychology · Interpretation (philosophy) · Interview · Respondent · Sociology · Statistics · Variety (cybernetics) · Web application · World Wide Web · Applied Psychology · Artificial Intelligence · Behavioral Health and Interventions · Computer Science · Psychology · Social and Intergroup Psychology · Social Psychology · Survey Methodology and Nonresponse
Do emotions affect political trust judgments?
Measurement invariance in the social sciences
Enhancing primary teachers’ formative assessment literacy
An experimental test of the effectiveness of cognitive interviewing in pretesting questionnaires
“I have to pick a percentage now”
Assessing conceptual comparability of single-item survey instruments with a mixed-methods approach
An empirical evaluation of probing questions investigating question comprehensibility in web surveys
Semi-Automated Nonresponse Detection for Open-Text Survey Data
How to Design, Implement, and Analyse a Survey
Combining numbers and perspectives
How Valid Are Trust Survey Measures? New Insights From Open-Ended Probing Data and Supervised Machine Learning
The Effects of Open-Ended Probes on Closed Survey Questions in Web Surveys
Cognitive Interviewing
Survey Questions
Social and Behavioral Research and the Internet
How participants report their health status
Evaluating political parties
The Length of Responses to Open-Ended Questions
Web Surveys
Testing the Use of Voice Input in a Smartphone Web Survey
Cognitive Probes in Web Surveys
The Influence of Topic Interest and Interactive Probing on Responses to Open-Ended Questions in Web Surveys
Item non-response in open-ended questions
The Effects of Embedding Closed-ended Cognitive Probes in a Web Survey on Survey Response
Computing Response Metrics for Online Panels
Open-Ended Questions in Web Surveys
Necessary but Insufficient
Do Open-Ended Questions Measure "Salient" Issues
Self-reports
Research Synthesis
The Random Probe
The Open and Closed Question
Comparing Traditional and Crowdsourcing Methods for Pretesting Survey Questions
Exploring Mental Models behind Self-rated Health and Subjective Life Expectancy through Web Probing
Comparing Cognitive Interviewing and Online Probing
How Many Interviews Are Enough
An Empirical Evaluation of In-Depth Probes Used to Pretest Survey Questions
| Unique citing works | 12 |
|---|---|
| Citations per year | 4 |
| Citation span | 2023 - 2026 (4) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 10 |