Intertemporal Differences Among MTurk Workers
Time-Based Sample Variations and Implications for Online Data Collection
Bibliographic Data
| ID | 16911908 |
|---|---|
| Authors | Linda Casey (0000-0002-8232-7925, Harvard University), Logan S Casey, Joan Chandler (0000-0001-8151-0915, University of Michigan, corresponding author), Jesse Chandler, Adam Seth Levine (0000-0003-4014-1003, Cornell University), A Proctor (0000-0002-2469-644X, Princeton University), Dara Z Strolovitch (0000-0002-1051-8036, Princeton University) |
| Year | 2017 |
| Volume | 7 |
| Issue | 2 |
| Publication date | 2017-04-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | SAGE Open (JOURNAL) |
| Journal identifiers | ISSN: 2158-2440 • E-ISSN: 2158-2440 |
| Publisher | SAGE Publications (PUBLISHER • US) |
| DOI | 10.1177/2158244017712774 |
| OpenAlex | W2625865465 |
| Language | EN |
| Citations received | 27 |
| References cited | 48 |
The online labor market Amazon Mechanical Turk (MTurk) is an increasingly popular source of respondents for social science research. A growing body of research has examined the demographic composition of MTurk workers as compared with that of other populations. While these comparisons have revealed the ways in which MTurk workers are and are not representative of the general population, variations among samples drawn from MTurk have received less attention. This article focuses on whether MTurk sample composition varies as a function of time. Specifically, we examine whether demographic characteristics vary by (a) time of day, (b) day of week, and serial position (i.e., earlier or later in data collection), both (c) across the entire data collection and (d) within specific batches. We find that day of week differences are minimal, but that time of day and serial position are associated with small but important variations in demographic composition. This demonstrates that MTurk samples cannot be presumed identical across different studies, potentially affecting reliability, validity, and efforts to reproduce findings
Amazon rainforest · Biology · Data collection · Population · Sociology · Statistics · Demography · Digital Marketing and Social Media · Mathematics · Psychology · Sharing Economy and Platforms · Technology Adoption and User Behaviour
An Oasis of Equality? Race and Stress on the MTurk Platform
When green energy feels cozy
Who prefers renewable energy? A moderated mediation model including perceived comfort and consumers’ protected values in green energy adoption and willingness to pay a premium
Melanin and Curls
Do Giving Circles Democratize Philanthropy? Donor Identity and Giving to Historically Marginalized Groups
Are Nonprobability Surveys Fit for Purpose
Online panels in social science research
Are Relational Inferences from Crowdsourced and Opt-in Samples Generalizable? Comparing Criminal Justice Attitudes in the GSS and Five Online Samples
MTurk Research
Believing in dissociative amnesia relates to claiming it
Remembering times ahead
“But not the music”
How Do Older Adults Recruited Using MTurk Differ From Those in a National Probability Sample
Crowdsourcing the Implicit Association Test
Privacy and Information Avoidance
The effects of Covid-19 on perceived intergroup competition and negative intergroup outcomes
Counting polyamorists who count
TikTok and Civic Activity Among Young Adults
Individual Utilities of Life Satisfaction Reveal Inequality Aversion Unrelated to Political Alignment
Group Selection and Opportunities for Gender Diversity in the Judiciary
Harnessing Romance
Get rich quick,” scheme or script? The effect of cryptoculture on the susceptibility of fraud victimization among cryptocurrency purchasers
So Help Me God’? Does oath swearing in courtroom scenarios impact trial outcomes
Crowdsourcing Reliable Local Data
How racialized policy contact shapes the social constructions of policy targets
Naturalise or deport? the distinct logics of support for different immigration outcomes
Effects of Participant Displeasure on the Social-Psychological Study of Power on Amazon's Mechanical Turk
Reputation as a sufficient condition for data quality on Amazon Mechanical Turk
The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers
Using Nonnaive Participants Can Reduce Effect Sizes
Using Mechanical Turk to Study Clinical Populations
Separate but equal? A comparison of participants and data gathered via Amazon’s MTurk, social media, and face-to-face behavioral testing
Nonnaïveté among Amazon Mechanical Turk workers
Conducting Clinical Research Using Crowdsourced Convenience Samples
The N-Pact Factor
Convenience samples of college students and research reproducibility
Prevalence of Experiences With Consensual Nonmonogamous Relationships
Inside the Turk
TurkPrime.com
Do Students with Different Characteristics Take Part in Psychology Experiments at Different Times of the Semester
Comparing Data Characteristics and Results of an Online Factorial Survey between a Population-Based and a Crowdsource-Recruited Sample
The Generalizability of Survey Experiments
Cross-Sample Comparisons and External Validity
Mobile Response in Web Panels
Not to Be Considered Harmful
Data Quality in PC and Mobile Web Surveys
Subjective measures of socio-economic position and the wealth index
A very brief measure of the Big-Five personality domains
Beyond the Turk
Improving Response Rates and Questionnaire Design for Mobile Web Surveys
Estimating Bias Due to Nonresponse in Mail Surveys
Personality and Legislative Politics
Web Surveys
Evaluating Online Labor Markets for Experimental Research
Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference
The Big Five Personality Traits in the Political Arena
Handbook of Personality Theory and Research
| Unique citing works | 27 |
|---|---|
| Citations per year | 3,86 |
| Citation span | 2019 - 2026 (8) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 23 |