False Positives and the “More-is-Better” Assumption in Sensitive Question Research
Bibliographic Data
| ID | 6370621 |
|---|---|
| Authors | Felix Wolter (0000-0002-0686-8527), A Diekmann (0000-0002-7344-5591) |
| Year | 2021 |
| Volume | 85 |
| Issue | 3 |
| Pages | 836-863 |
| Publication date | 2021-12-03 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Public Opinion Quarterly (JOURNAL) |
| Journal identifiers | ISSN: 0033-362X • E-ISSN: 1537-5331 |
| Publisher | Oxford University Press (OUP) (PUBLISHER) |
| DOI | 10.1093/poq/nfab043 |
| OpenAlex | W3215582806 |
| Language | EN |
| Citations received | 3 |
| References cited | 24 |
Several special questioning techniques have been developed in order to counteract misreporting to sensitive survey questions, for example, on criminal behavior. However, doubts have been raised concerning their validity and practical value as well as the strategy of testing their validity using the “more-is-better” assumption in comparative survey experiments. This is because such techniques can be prone to generating false positive estimates, that is, counting “innocent” respondents as “guilty” ones. This article investigates the occurrence of false positive estimates by comparing direct questioning, the crosswise model (CM), and the item count technique (ICT). We analyze data from two online surveys (N = 2,607 and 3,203) carried out in Germany and Switzerland. Respondents answered three questions regarding traits for which it is known that their prevalence in reality is zero. The results show that CM suffers more from false positive estimates than ICT. CM estimates amount to up to 15 percent for a given true value of zero. The mean of the ICT estimates is not significantly different from zero. We further examine factors causing the biased estimates of CM and show that speeding through the questionnaire (random answering) and problems with the measurement procedure—namely regarding the unrelated questions—are responsible. Our findings suggest that CM is problematic and should not be used or evaluated without the possibility of accounting for false positives. For ICT, the issue is less severe
Econometrics · False positive paradox · False positives and false negatives · Information and Communications Technology · Political science · Statistics · Value (mathematics · Zero (linguistics · Hate Speech and Cyberbullying Detection · Mathematics · Psychology · Survey Methodology and Nonresponse · Survey Sampling and Estimation Techniques · Law · Social Psychology
Combining List Experiment and Direct Question Estimates of Sensitive Behavior Prevalence
Randomized Response
A new scale of social desirability independent of psychopathology.
Response strategies for coping with the cognitive demands of attitude measures in surveys
Meta-Analysis of Randomized Response Research
Making Use of “Benford’s Law” for the Randomized Response Technique
Uncovering a Blind Spot in Sensitive Question Research
When to Protect? Using the Crosswise Model to Integrate Protected and Direct Responses in Surveys of Sensitive Behavior
Asking Sensitive Questions Using the Crosswise Model
Social desirability bias in voter turnout reports
Sensitive Questions in Surveys
Asking the Embarrassing Question
Placebo statements in list experiments
Artificial Inflation or Deflation? Assessing the Item Count Technique in Comparative Surveys
Statistical Analysis of List Experiments
The Misreporting Trade-Off Between List Experiments and Direct Questions in Practice
List Experiment Design, Non-Strategic Respondent Error, and Item Count Technique Estimators
When to Worry about Sensitivity Bias
| Citations per year | 1 |
|---|---|
| Citation span | 2023 - 2026 (4) |
| Highly cited | No |