Data Exclusion in Policy Survey and Questionnaire Data

Data preprocessing is an integral step prior to analyzing data in psychological science, with implications for its potentially guiding policy. This article reports how psychological researchers address data preprocessing or quality concerns, with a focus on aberrant responses and missing data in self-report measures. 240 articles were sampled from four journals: Psychological Science, Journal of Personality and Social Psychology, Developmental Psychology, and Abnormal Psychology from 2012 to 2018. Nearly half of the studies did not report any missing data treatment (111/240; 46.25%), and if they did, the most common approach was listwise deletion (71/240; 29.6%). Studies that remove data due to missingness removed, on average, 12% of the sample. Likewise, most studies do not report any aberrant responses (194/240; 80%), but if they did, they classified 4% of the sample as suspect. Most studies are either not transparent enough about their data preprocessing steps or may be leveraging suboptimal procedures. Recommendations can improve transparency and data quality

Criminology · Data mining · Data pre-processing · Data quality · Data science · Machine learning · Missing data · Preprocessor · Psychological science · Statistics · Survey data collection · Suspect · Applied Psychology · Computer Science · Data-Driven Disease Surveillance · Economic and Environmental Valuation · Mathematics · Psychology · Social Psychology · Survey Methodology and Nonresponse · Artificial Intelligence

• Public attitudes and values regarding a semi‐urban feral ungulate

  Open Access•Danhe Yang, Debottam Bhattacharjee et al.•People and Nature•2025

  Añadir item

• Validation of a Chinese version of the Nature Relatedness Scale (NRS-C)

  Open Access•Chunrong Zuo, Nor Shafrin Ahmad et al.•Acta Psychologica•2025

  Añadir item

• Missing Data

  Open Access•Philip L Roth•Personnel Psychology•1994

  Añadir item

• Methods to detect low quality data and its implication for psychological research

  Open Access•Erin M Buchanan, John E Scofield•Behavior Research Methods•2018

  Añadir item

• Missing Data in Educational Research

  Open Access•James L Peugh, Craig K Enders•Review of Educational Research•2004

  Añadir item

• Inference and missing data

  Donald B Rubin•Biometrika•1976

  Añadir item

• The Relative Performance of Full Information Maximum Likelihood Estimation for Missing Data in Structural Equation Models

  Craig K Enders, Craig Enders et al.•Structural Equation Modeling: A…•2001

  Añadir item

• Running experiments on Amazon Mechanical Turk

  Open Access•Gabriele Paolacci, Joan Chandler et al.•Judgment and Decision Making•2010

  Añadir item

• False-Positive Psychology

  Open Access•Joseph P Simmons, L D Nelson et al.•Psychological Science•2011

  Añadir item

• Ensuring survey research data integrity in the era of internet bots

  Open Access•Marybec Griffin, Richard J Martino et al.•Quality & Quantity•2022

  Añadir item

• An MTurk Crisis? Shifts in Data Quality and the Impact on Study Results

  Open Access•Michael Chmielewski, Michael S Chmielewski et al.•Social Psychological and…•2020

  Añadir item

• The Performance of the Full Information Maximum Likelihood Estimator in Multiple Regression Models with Missing Data

  Open Access•Craig K Enders•Educational and Psychological…•2001

  Añadir item

• Methods of Detecting Insufficient Effort Responding

  Open Access•Maxwell Hong, Jeffrey T Steedle et al.•Educational and Psychological…•2020

  Añadir item

• Detecting and Deterring Insufficient Effort Responding to Surveys

  Open Access•Jason L Huang, Paul Curran et al.•Journal of Business and Psychology•2011

  Añadir item

• Methods for the detection of carelessly invalid responses in survey data

  Open Access•Paul Curran, Paul G Curran•Journal of Experimental Social…•2015

  Añadir item

• Should We Trust Web-Based Studies? A Comparative Analysis of Six Preconceptions About Internet Questionnaires

  Samuel D Gosling, Simine Vazire et al.•American Psychologist•2004

  Añadir item

• Statistical Analysis with Missing Data

  Open Access•Bo Ning, N Bo et al.•Population•1988

  Añadir item