Response Speed and Response Consistency as Mutually Validating Indicators of Data Quality in Online Samples
Bibliographic Data
| ID | 21208613 |
|---|---|
| Authors | Dustin Wood (0000-0003-1625-0656, University of Alabama, corresponding author), Peter D Harms (0000-0003-4477-6313, University of Alabama), Graham H Lowman (0000-0003-2528-070X, University of Alabama), Justin A DeSimone (0000-0002-2471-6834, University of Alabama) |
| Year | 2017 |
| Volume | 8 |
| Issue | 4 |
| Pages | 454-464 |
| Publication date | 2017-05-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Social Psychological and Personality Science (JOURNAL) |
| Journal identifiers | ISSN: 1948-5506 • E-ISSN: 1948-5514 |
| Publisher | SAGE Publications (PUBLISHER • US) |
| DOI | 10.1177/1948550617703168 |
| OpenAlex | W2621532439 |
| Language | EN |
| Citations received | 67 |
| References cited | 35 |
We examine the appropriateness of response speed and response consistency as data quality indicators within online samples. Across several inventories, results show that response consistency decreases dramatically at response rates faster than 1 second per item. Our results suggest that careless responding may be fairly common in online samples and often functions to increase the expected correlation between items in a survey, which has implications for the likelihood of false positives and the analysis of factor structure. Given how careless responding can influence estimated associations between variables, we strongly recommend that researchers include response speed and consistency screens in their research and provide empirically informed cut points for data screens that should be useful across a wide range of instruments and settings
Data consistency · Data quality · Econometrics · False positive paradox · Item response theory · Psychometrics · Response bias · Statistics · Behavioral Health and Interventions · Clinical Psychology · Computer Science · Digital Marketing and Social Media · Mathematics · Psychology · Social and Intergroup Psychology · Social Psychology · Artificial Intelligence
Building evidence for indoor tanning as a behavioral addiction
Estimating the reliability of emotion measures over very short intervals
Long live A(me)rica! An examination of the interplay between nationalistic–symbolic immortality striving and belief in life after death
Reliability from α to ω
Data quality in online human-subjects research
Online panels in social science research
MTurk Research
Innovations in Sampling
International Prevalence and Correlates of Psychological Stress during the Global Covid-19 Pandemic
A Clarified Examination of the Item Wording Effect
Chatbots Are Undermining Crowdsourced Research in the Behavioral Sciences
Value and moral divergences between management researchers and managers
Designing Eco‐Friendly Message Sidedness for Consumer Happiness
Is It Fair to be Accurate? Moral-Emotional Responses to Organizations’ AI Orientation Choices
Mathematics Teacher Leadership
The relationship between gratitude and life satisfaction
The prevalence of careless response behaviour and its consequences on data quality in self-report questionnaires on student learning
Are “justified” biases acceptable? Citizens’ perceptions of gendered government hiring
Predictors of enhancing human physical attractiveness
Testing the generalizability of the white leadership standard in the post-Obama era
Understanding left-wing authoritarianism
Different developmental feedback, same employee performance improvement
Moral framing effects within subjects
Trust erodes, fatigue builds
Partisan Prejudice
Drivers of Data Quality in Advertising Research
Crowdsourcing the Implicit Association Test
Crime severity, relational distance, and bystander reporting
Test-Retest Reliability and Construct Validity of the Brief Dark Triad Measurements
Developing and Improving Personality Inventories Using Generative Artificial Intelligence
The threat of electronic performance monitoring
The inter-relationships between sensory processing sensitivity and mental health problems among college students
Anti-noise window
An MTurk Crisis? Shifts in Data Quality and the Impact on Study Results
How Dropping Subjects Who Failed Manipulation Checks Can Bias Your Results
Distinguishing between individual and societal socioeconomic mobility beliefs
Can Overclaiming Technique Improve Self-Assessment Tools for Digital Competence? The Case of DigCompSat
Motivational climates, work passion, and behavioral consequences
Learning From Mistakes
Influencing factors of nurses’ short-term bereavement reactions after patient death
Gender Identity Awareness as a Shadow Pillar of the Gender Hierarchy
Getting away “Scott” (but not Susan) free
Meating of the minds
Beyond message framing
Addressing fraudulent responses in online surveys
Wording Effects in Assessment
Examining the Lower‐Order Structure of Openness/Intellect Using Traditional and Extended Bass‐Ackwards Methods
Resilience of primal world beliefs to the initial shock of the Covid ‐19 pandemic
Differential responses to ethical vegetarian appeals
Providing feedback without verification
Discovering Preference Structure Using Randomized Paired Comparisons in Surveys
Does the military lose public confidence without compliance with civilian control? Experimental evidence from Japan
Believing in the American Dream Sustains Negative Attitudes toward Those in Poverty
Using multi-rater and test-retest data to detect overlap within and between psychological scales
Curiosity as feelings of interest versus deprivation
Unintended effects of measuring implicit processes
How morality signals, benefits, binds, and teaches
Defending one's worldview under mortality salience
Pedophile, Child Lover, or Minor-Attracted Person? Attitudes Toward Labels Among People Who are Sexually Attracted to Children
Surviving the screens
Racing the Clock
Instrumental Motives in Emotion Regulation of One’s Own and Others’ Anger
Gender-Based Evaluations Of Integrity Failures Women Leaders Judged Worse
Measuring Attentiveness in Self-Administered Surveys
Analyze the attentive and bypass bias
Online Nonprobability Samples
Games in Socioenvironmental Research
Internal Consistency, Retest Reliability, and Their Implications for Personality Scale Validity
Locally Weighted Regression
Insufficient effort responding
The dirty dozen
Introducing the Short Dark Triad (SD3)
Factors Defined by Negatively Keyed Items
Nonnaïveté among Amazon Mechanical Turk workers
The Panas-X
A Basis for Analyzing Test-Retest Reliability
Careless Responding to Reverse-Worded Items
The Correlates of Similarity Estimates Are Often Misleadingly Positive
Amazon's Mechanical Turk
Random Responding as a Threat to the Validity of Effect Size Estimates in Correlational Research
Survey research methods
The viability of crowdsourcing for survey research
Detecting and Deterring Insufficient Effort Responding to Surveys
Years of higher-order confirmatory factor analysis in the organizational sciences
Best practice recommendations for data screening
Ascertaining the validity of individual protocols from Web-based personality inventories
Identification and measurement of a more comprehensive set of person-descriptive trait markers from the English lexicon
The Dark Triad of personality
Should We Trust Web-Based Studies? A Comparative Analysis of Six Preconceptions About Internet Questionnaires
Continuity and change
The nature and structure of correlations among Big Five ratings
The next Big Five Inventory (BFI-2)
The developmental psychometrics of big five self-reports
The pitfall of experimenting on the web
Standard Operating Procedures
| Unique citing works | 67 |
|---|---|
| Citations per year | 7,44 |
| Citation span | 2017 - 2026 (10) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 64 |