Skip to main content

ETHNOS_APP

Home • Search • Journals • List 0

Factors Defined by Negatively Keyed Items

The Result of Careless Respondents?

Bibliographic Data

ID23345977
AuthorsNeal Schmitt (0000-0002-2806-348X, Michigan State University), Daniel M Stuits (Quaker Oats Company)
Year1985
Volume9
Issue4
Pages367-373
Publication date1985-12-01
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueApplied Psychological Measurement (JOURNAL)
Journal identifiersISSN: 0146-6216 • E-ISSN: 1552-3497
PublisherSAGE Publications (PUBLISHER • US)
DOI10.1177/014662168500900405
OpenAlexW2129501685
LanguageEN
Citations received105
References cited23

A frequently occurring phenomenon in factor and cluster analysis of personality or attitude scale items is that all or nearly all questionnaire items that are nega tively keyed will define a single factor. Although sub stantive interpretations of these negative factors are usually attempted, this study demonstrates that the negative factor could be produced by a relatively small portion of the respondents who fail to attend to the negative-positive wording of the items. Data were generated using three different correlation matrices, which demonstrated that regardless of data source, when only 10% of the respondents are careless in this fashion, a clearly definable negative factor is gener ated. Recommendations for instrument development and data editing are presented.

Cluster (spacecraft) · Correlation · Factor (programming language) · Personality · Scale (ratio) · Statistics · Computer Science · Consumer Behavior in Brand Consumption and Identification · Eating Disorders and Behaviors · Mathematics · Psychology · Sensory Analysis and Statistical Methods · Social Psychology

  • The reliability of the twelve-item general health questionnaire (GHQ-12) under realistic assumptions

    Open Access•Matthew Hankins•BMC Public Health•2008

  • Reversed item bias

    Bert Weijters, Hans Baumgartner et al.•Psychological Methods•2013

  • Is It Biased? Empirical Analysis of Various Phenomena That Affect Survey Results

    Open Access•Luka Mandić, Ksenija Klasnić•Revija za sociologiju•2021

  • Escala de Necessidade de Cognição (NCS-18)

    Open Access•Valdiney V Gouveia, Luis Augusto De Carvalho Mendes et al.•Psicologia Reflexão e Crítica•2015

  • The ubiquity of common method variance

    Open Access•Michael D Biderman, Nhung Nguyen et al.•Journal of Research in Personality•2011

  • An evaluation of the consequences of using short measures of the Big Five personality traits

    Marcus Credé, P D Harms et al.•Journal of Personality and Social…•2012

  • The behavioral ecology of moral dilemmas

    Heather M Maranges, Connor R Hasty et al.•Journal of Personality and Social…•2021

  • Reconceptualizing Anxiety as a Continuum That Ranges From High Calmness to High Anxiety

    Open Access•Andy P Siddaway, Paul J Taylor et al.•Journal of Personality and Social…•2017

  • Content validation guidelines

    Jason A Colquitt, Tyler B Sabey et al.•Journal of Applied Psychology•2019

  • Differences Between Tight and Loose Cultures

    Open Access•Michele J Gelfand, Jana L Raver et al.•Science•2011

  • Measuring Personality and Psychopathology

    Open Access•Leslie C Morey•Handbook of Psychology, Second…•2012

  • Measuring self-concept across the life span

    Barbara M Byrne•Measuring self-concept across the…•1996

  • Cognitive versus affective determinants of organizational citizenship behavior.

    Dennis W Organ, Mary A Konovsky et al.•Journal of Applied Psychology•1989

  • How Should the Internal Structure of Personality Inventories Be Evaluated?

    Open Access•Christopher J Hopwood, M Brent Donnellan•Personality and Social Psychology…•2010

  • Development of the short version of the spielberger state—trait anxiety inventory

    Open Access•Andras N Zsido, Szidalisz Ágnes Teleki et al.•Psychiatry Research•2020

  • A Review of Scale Development Practices in the Study of Organizations

    Open Access•Timothy R Hinkin•Journal of Management•1995

  • Cultural Intelligence

    Open Access•Ang, Soon Ang et al.•Management and Organization Review•2007

  • Measuring Public Service Motivation

    Bradley E Wright, Robert K Christensen et al.•International Public Management…•2013

  • Transformational Leadership and Job Behaviors

    Ronald F Piccolo, Jason A Colquitt•Academy of Management Journal•2006

  • Integrating the Bright and Dark Sides of OCB

    Joel Koopman, Klodiana Lanaj et al.•Academy of Management Journal•2016

  • Are Attention Check Questions a Threat to Scale Validity?

    Open Access•Franki Y H Kung, Navio Kwok et al.•Applied Psychology•2018

  • Assessing Three Sources of Misresponse to Reversed Likert Items

    Scott D Swain, Danny Weathers et al.•Journal of Marketing Research•2008

  • The effect of rating scale format on response styles

    Open Access•Bert Weijters, Elke Cabooter et al.•International Journal of Research…•2010

  • Amazon Mechanical Turk in Organizational Psychology

    Open Access•Janelle H Cheung, Deanna K Burns et al.•Journal of Business and Psychology•2017

  • Guidelines for developing, translating, and validating a questionnaire in perioperative and pain medicine

    Open Access•Siny Tsang, Colin Royse et al.•Saudi Journal of Anaesthesia•2017

  • Screening for Dark Personalities

    Open Access•Delroy L Paulhus, Erin E Buckels et al.•European Journal of Psychological…•2021

  • Ineffectiveness of Reverse Wording of Questionnaire Items

    Open Access•Eric Van Sonderen, R Sanderman et al.•PLoS ONE•2013

  • Resilience Building in Students

    Open Access•Cassidy, Simon Cassidy•Frontiers in Psychology•2015

  • Misresponse to Reversed and Negated Items in Surveys

    Open Access•Bert Weijters, Hans Baumgartner•Journal of Marketing Research•2012

  • Scale Construction

    Open Access•Timothy R Hinkin, J Bruce Tracey et al.•Journal of Hospitality & Tourism…•1997

  • Dirty Data

    Open Access•Justin A DeSimone, Peter D Harms•Journal of Business and Psychology•2018

  • Careless Responding to Reverse-Worded Items

    Open Access•Carol M Woods•Journal of Psychopathology and…•2006

  • Assessing callous–unemotional traits in adolescent offenders

    Open Access•Eva R Kimonis, Paul J Frick et al.•International Journal of Law and…•2008

  • The System Usability Scale

    James R Lewis•International Journal of…•2018

  • A Brief Tutorial on the Development of Measures for Use in Survey Questionnaires

    Open Access•Timothy R Hinkin•Organizational Research Methods•1998

  • Detecting careless respondents in web-based questionnaires

    Open Access•A Susan M Niessen, Rob R Meijer et al.•Journal of Research in Personality•2016

  • The Lazy or Dishonest Respondent

    Winfred Arthur, Ellen Hagen et al.•Annual Review of Organizational…•2020

  • Psychometric evaluation of the German version of the Opening Minds Stigma Scale for Health Care Providers (OMS-HC)

    Open Access•Gianfranco Zuaboni, Timon Elmer et al.•BMC Psychology•2021

  • Direction of Wording Effects in Balanced Scales

    Open Access•Timothy R Miller, T Anne Cleary•Educational and Psychological…•1993

  • Psychometric evaluation of the Swedish version of the pure procrastination scale, the irrational procrastination scale, and the susceptibility to temptation scale in a clinical population

    Open Access•Alexander Rozental, Erik Forsell et al.•BMC Psychology•2014

  • Nonjudgmental Regard of Others

    Open Access•Adelheid A M Nicol, Kalee De France•Psychological Reports•2022

  • Urban Public Spaces as Restorative Environments

    Open Access•Katarina Polajnar Horvat, Daniela Ribeiro•International Journal of…•2023

  • Item Reversals and Response Validity in the Job Diagnostic Survey

    Open Access•Beverly G Burke•Psychological Reports•1999

  • Item Reversals and Response Validity in the Job Diagnostic Survey

    Beverly G Burke•Psychological Reports•1999

  • A Clarified Examination of the Item Wording Effect

    Open Access•Zdravko Marjanovic, Anna Louisa Maidens•Psychological Reports•2024

  • Inconsistent Responding on a Mixed-Worded Scale in the Pisa 2022 Questionnaire

    Open Access•Michalis P Michaelides, Evi Konstantinidou et al.•Journal of Psychoeducational…•2026

  • Assessment mode and inconsistent responding on a mixed-worded scale

    Open Access•Evi Konstantinidou, Michalis P Michaelides•Frontiers in Education•2025

  • Why Do Students Attend STEM Clubs, What Do They Get Out of It, and Where Are They Heading

    Open Access•Margaret R Blanchard, Kristie Gutierrez et al.•Education Sciences•2023

  • Effect of Method Bias on the Internal Structure of the Systemic Clinical Outcomes in Routine Evaluation Measure of Family Functioning

    Open Access•Christina Hamme Peterson, N Andrew Peterson et al.•The Family Journal•2023

  • An item response theory analysis of Rosenberg Self-Esteem among older Korean adults

    Open Access•Kyeongwon Lee•Current Psychology•2023

  • Validation of the motivated strategies for learning questionnaire and instructional materials motivation survey

    Donald A Cook, Lee P Skrupky•Medical Teacher•2025

  • Factorial Structure and Nomological Network of the Inventory of Callous-Unemotional Traits Accounting for Item Keying Variance

    Chrystalla C Koutsogiorgi, Chrystalla Koutsogiorgi et al.•Journal of Personality Assessment•2021

  • Reattachment to work in the morning and day-level leader outcomes

    Open Access•Charlotte Fritz, Dana Auten et al.•Journal of Vocational Behavior•2021

  • Response Speed and Response Consistency as Mutually Validating Indicators of Data Quality in Online Samples

    Open Access•Dustin Wood, Peter D Harms et al.•Social Psychological and…•2017

  • Investigation of factors underlying foreign language classroom anxiety in Chinese university English majors

    Shuting Zhang, Chun Lai•Applied Linguistics Review•2024

  • Challenges in Measuring a New Construct

    Open Access•Victoria A Miller, William W Reynolds et al.•Journal of Empirical Research on…•2009

  • In the Eyes of the Beholder

    Open Access•Janaki Gooty, Mark Gavin et al.•Journal of Leadership &…•2009

  • Re-Examining The Validity of Hofstede's Power Distance Dimension

    Open Access•Jung-Soo Yi•Journal of Intercultural…•2026

  • Why don’t we care more about carelessness? Understanding the causes and consequences of careless participants

    Austin Nichols, Austin Lee Nichols et al.•International Journal of Social…•2020

  • Leader–member exchange differentiation and employee performance

    Open Access•Haesang Park, Haeseen Park et al.•Journal of Organizational Behavior•2022

  • Memento Mori

    Open Access•Zhenyu Yuan, Lisa E Baranik et al.•Journal of Organizational Behavior•2018

  • The dual commitments of contingent workers

    Open Access•Robert C Liden, Stephen J Wayne et al.•Journal of Organizational Behavior•2003

  • Best practice recommendations for scale construction in organizational research

    Open Access•Thomas A Wright, James Campbell Quick et al.•Journal of Organizational Behavior•2017

  • Breaking rules for the right reasons? An investigation of pro‐social rule breaking

    Open Access•Jason J Dahling, Samantha Le Chau et al.•Journal of Organizational Behavior•2010

  • The relationship between ethical leadership and core job characteristics

    Open Access•Ronald F Piccolo, Rebecca L Greenbaum et al.•Journal of Organizational Behavior•2010

  • The Community Readiness Instrument

    Sijia Zhang, Natalie M Ricciutti et al.•Measurement and Evaluation in…•2025

  • The Spiritual Well-Being Scale

    Open Access•Klara Malinakova, Jaroslava Kopcakova et al.•Journal of Religion and Health•2017

  • An Experimental Approach to Evaluate Methods to Detect Scale Recalibration

    Open Access•M Janell Granier, Samuel B Green et al.•Educational and Psychological…•1991

  • A Confirmatory Factor Analysis Examination of Reverse Coding Effects in Meyer and Allen's Affective and Continuance Commitment Scales

    Open Access•Sherry L Magazine, Larry J Williams et al.•Educational and Psychological…•1996

  • Validity Evidence for the Organizational Commitment Questionnaire in the Japanese Corporate Culture

    Open Access•Marion M White, Judi McLean Parks et al.•Educational and Psychological…•1995

  • The Impact of Positive and Negative Item Stems on the Validity of a Computer Anxiety Scale

    Open Access•William J Pilotte, Robert K Gable•Educational and Psychological…•1990

  • Developing an Interpretation of Item Parameters for Personality Items

    Open Access•Michael J Zickar, Karen L Ury•Educational and Psychological…•2002

  • Detecting Careless Responding in Multidimensional Forced-Choice Questionnaires

    Open Access•Rebekka Kupffer, Susanne Frick et al.•Educational and Psychological…•2024

  • An IRT Examination of the Psychometric Functioning of Negatively Worded Personality Items

    Open Access•Katherine A Sliter, Michael J Zickar•Educational and Psychological…•2014

  • Improving the Factor Structure of Psychological Scales

    Open Access•Xijuan Zhang, Victoria Savalei•Educational and Psychological…•2016

  • Comparing the Psychometric Properties of a Scale Across Three Likert and Three Alternative Formats

    Open Access•Xijuan Zhang, Linnan Zhou et al.•Educational and Psychological…•2023

  • A Longitudinal Examination of the Dimensionality and Stability of the Organizational Commitment Questionnaire (OCQ)

    Open Access•Lois E Tetrick, Arthur J Farkas•Educational and Psychological…•1988

  • The Effect of Negation and Polar Opposite Item Reversals on Questionnaire Reliability and Validity

    Open Access•Chester A Schriesheim, Regina J Eisenbach et al.•Educational and Psychological…•1991

  • Perceptions of mental health professionals and family caregivers about their collaborative relationships

    Open Access•Megan Andrew, John Farhall et al.•Australian Psychologist•2009

  • Measuring students' AI competence

    Open Access•Zubair Ahmad, Almaas Sultana et al.•Acta Psychologica•2025

  • Detecting and Deterring Insufficient Effort Responding to Surveys

    Open Access•Jason L Huang, Paul Curran et al.•Journal of Business and Psychology•2011

  • Self-assessment of second language ability

    Open Access•L Kathy Heilenman•Language Testing•1990

  • Acquiescence as differential person functioning

    Gunnar Johanson, GEORGE A JOHANSON et al.•Assessment & Evaluation in Higher…•2004

  • Shifting gears

    Michael J Roszkowski, Margot Soven•Assessment & Evaluation in Higher…•2010

  • The Impact of Item Feature and Response Preference in a Mixed-Format Design

    Hui-Fang Chen, Kuan‐Yu Jin•Multivariate Behavioral Research•2022

  • A Systematic Evaluation of Wording Effects Modeling Under the Exploratory Structural Equation Modeling Framework

    Luís Eduardo Garrido, Alexander P Christensen et al.•Multivariate Behavioral Research•2025

  • The importance of being … social? Instructor credibility and the Millennials

    Megan W Gerhardt•Studies in Higher Education•2016

  • Preventing satisficing

    Danielle R Blazek, James T Siegel•International Journal of Social…•2024

  • Temel Psikolojik İhtiyaçlar Ölçeği-Revize Formu

    Open Access•Asiye Yıldırım•Afyon Kocatepe Üniversitesi…•2024

  • Deaf and Hard-of-Hearing Students' Experiences in Mainstream and Separate Postsecondary Education

    John T E Richardson, G Venkatesulu et al.•The Journal of Deaf Studies and…•2010

  • What Shall We Do About Grit? A Critical Review of What We Know and What We Don’t Know

    Open Access•Marcus Credé•Educational Researcher•2018

  • Experiencing culture in attractions, events and tour settings

    Open Access•Greg Richards, B King et al.•Tourism Management•2020

  • Use of food attitudes and behaviors in determination of the personality characteristic of openness

    Open Access•Lakshman Rajagopal, Fayrene L Hamouz•International Journal of…•2009

  • Differentiating conscientious from indiscriminate responders in existing NEO-Five Factor Inventory-3 data

    Open Access•Zdravko Marjanovic, Ronald R Holden•Journal of Research in Personality•2019

  • Implications of Item Wording for Hardiness Structure, Relation with Neuroticism, and Stress Buffering

    Open Access•Robert R Sinclair, Lois E Tetrick•Journal of Research in Personality•2000

  • The Factor Structure of Global Self-Esteem among Adolescents and Adults

    Open Access•Robert Flynn Corwyn•Journal of Research in Personality•2000

  • Measuring the Intrapersonal Component of Psychological Empowerment

    Open Access•N Andrew Peterson, John B Lowe et al.•American Journal of Community…•2006

  • Methods for the detection of carelessly invalid responses in survey data

    Open Access•Paul Curran, Paul G Curran•Journal of Experimental Social…•2015

  • The Organizational Survey Process

    Open Access•Jack E Edwards, Jack Edwards et al.•American Behavioral Scientist•1993

  • Questionário de Saúde Geral (QSG-12)

    Open Access•Valdiney V Gouveia, Tiago Jessé Souza Lima et al.•Cadernos de Saude Publica•2012

  • Society and the Adolescent Self-Image

    Morris Rosenberg•Society and the Adolescent…•1965

  • The authoritarian personality

    Theodor W Adorno•The authoritarian personality•1969

  • Acquiescence and Desirability As Response Determinants On the MMPI

    Open Access•Douglas N Jackson, Samuel Messick•Educational and Psychological…•1961

  • Acquiescence in the MMPI

    Open Access•Leonard G Rorer, Lewis R Goldberg•Educational and Psychological…•1965

  • The Measurement of Authoritarian Attitudes

    Open Access•Samuel Messick, Douglas N Jackson•Educational and Psychological…•1958

  • Workers' Central Life Interests and Job Performance

    Open Access•Robert Dubin, Joseph E Champoux•Sociology of Work and Occupations•1974

  • Psychological testing

    Frederick B Davis•Psychological Bulletin•1950

  • The great response-style myth

    Leonard G Rorer•Psychological Bulletin•1965

  • Response styles on the MMPI

    Douglas N Jackson, Samuel Messick•Journal of Abnormal & Social…•1962

  • Response set in the F scale

    Loren J Chapman, Donald T Campbell•Journal of Abnormal & Social…•1957

  • The Scree Test For The Number Of Factors

    Raymond B Cattell•Multivariate Behavioral Research•1966

  • Identification of content and style

    Peter M Bentler, Douglas N Jackson et al.•Psychological Bulletin•1971

  • On further conjectures regarding acquiescence

    Jack Block•Psychological Bulletin•1971

  • Industrial Workers' Worlds

    Robert Dubin•Social Problems•1956

Unique citing works105
Citations per year2,76
Citation span1988 - 2026 (39)
Citation velocitycurrent
Highly citedYes
Citation typesNeutral: 96

Tools

Open DOISci-HubOpen Access
Ethnos_APP • Open Source Project • MIT License • Frontend v2.0.0 • Privacy and Cookies • API Documentation: api.ethnos.app/docs • API Source Code: GitHub • DOI: 10.5281/zenodo.17049435 • Frontend Source Code: GitHub • DOI: 10.5281/zenodo.17050053 • cruz.rio.br • Expectantes Misericordiae