Measurement Schmeasurement
Questionable Measurement Practices and How to Avoid Them
Bibliographic Data
| ID | 23355293 |
|---|---|
| Authors | Jessica Flake (0000-0002-3498-615X, McGill University, corresponding author), Eiko I Fried (0000-0001-7469-594X, Leiden University) |
| Year | 2020 |
| Volume | 3 |
| Issue | 4 |
| Pages | 456-465 |
| Publication date | 2020-12-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Advances in Methods and Practices in Psychological Science (JOURNAL) |
| Journal identifiers | ISSN: 2515-2459 • E-ISSN: 2515-2467 |
| Publisher | SAGE Publications (PUBLISHER • US) |
| DOI | 10.1177/2515245920952393 |
| OpenAlex | W3108041100 |
| Language | EN |
| Citations received | 165 |
| References cited | 44 |
In this article, we define questionable measurement practices (QMPs) as decisions researchers make that raise doubts about the validity of the measures, and ultimately the validity of study conclusions. Doubts arise for a host of reasons, including a lack of transparency, ignorance, negligence, or misrepresentation of the evidence. We describe the scope of the problem and focus on how transparency is a part of the solution. A lack of measurement transparency makes it impossible to evaluate potential threats to internal, external, statistical-conclusion, and construct validity. We demonstrate that psychology is plagued by a measurement schmeasurement attitude: QMPs are common, hide a stunning source of researcher degrees of freedom, and pose a serious threat to cumulative psychological science, but are largely ignored. We address these challenges by providing a set of questions that researchers and consumers of scientific research can consider to identify and avoid QMPs. Transparent answers to these measurement questions promote rigorous research, allow for thorough evaluations of a study’s inferences, and are necessary for meaningful replication studies.
Computer security · Construct (python library) · Data science · External validity · Ignorance · Misrepresentation · Political science · Replication (statistics) · Scope (computer science) · Set (abstract data type) · Transparency (behavior) · Computer Science · Law · Mathematics · Mental Health Research Topics · Meta-analysis and systematic reviews · Psychology · Psychometric Methodologies and Testing · Social Psychology
Spillover and Crossover Effects of Financial Stress in Romantic Dyads
No Validity Without Reliability
Does Order Matter? The Impact of Methodological Decisions in Survey Research Concurrently Examining Attachment to Mothers and Fathers
The hidden universe of operationalizing natural experiments
Beyond the Qualitative-Quantitative Antinomy
Improving the Credibility of Empirical Legal Research
Aspiring to greater intellectual humility in science
Inflexitext
Measuring ACT in context
Perceived Addiction Potential
Thinking twice about sum scores
Measuring memory is harder than you think
Improving Social Science
Exploring student information problem solving behaviour using fine-grained concept map and search tool data
Psychometric Network Models from Time-Series and Panel Data
Capturing Environmental Dimensions of Adversity and Resources in the Context of Poverty Across Infancy Through Early Adolescence
Sameness and Difference in Psychological Research on Consensually Non-Monogamous Relationships
Dehumanization and Socioeconomic Status
Breadmakers and Breadwinners
Do smartphone usage scales predict behavior?
Normalizing the Use of Single-Item Measures
Defining and measuring “psychological flexibility”
Validating the Short-form Foreign Language Classroom Anxiety Scale
Understanding Research Engagement Among Academics in the Philippines Using the Theory of Planned Behavior
Estimating a minimum clinically important difference for the Developmental Behaviour Checklist – parent report
Social activities in multidomain dementia prevention interventions
Assessing and accounting for measurement in intensive longitudinal studies
Forwarding the ACBS Task Force recommendations
Examining item content across nine psychological (in)flexibility scales
Testing the Motivational Theory of Role Modeling in First- and Continuing-Generation College Students
Is the AAQ-II that bad
Does One Treatment Fit All? Effectiveness of a Multicomponent Cognitive Behavioral Therapy Program in Data-Driven Subtypes of Perinatal Depression
Automated Algorithmic Data Analysis for Confirmatory Factor Analysis and Invariance Assessment
On the measurement of relational responding
Mapping Methodological Variation in Experience-Sampling Research From Design to Data Analysis
Realizing the Full Potential of Big-Team Behavioral Science
Measurement-Reporting Practices in Social-and-Personality-Psychology Articles
Measuring Emotional Experiences with Music
Scientific reasoning scale in Italy
Application and effectiveness analysis of AI Intelligent tutoring system combined with knowledge graph in basic medical pathology teaching
Measures of public attitudes towards science
On the problems of construct proliferation and redundancy
Beyond Static Markers in Disproportionality Research
Stimulus-category confounds reduce, but do not explain, different roles of intent across moral domains
Exploring the measurement of political trust A multilevel observational analysis of six Swedish public agencies
Measuring authentic living from internal and external perspectives
Integration of pedagogical and psychometric approaches to assessing the social adaptation of schoolchildren
An Item Response Theory Analysis of the “Reading the Mind in the Eyes Test” in an Argentine Sample
Limitations of the Sum-and-Alpha Approach to Measurement in Behavioral Research
Household earthquake readiness
Communicating What We Know and What Isn’t So
A Novel, Network-Based Approach to Assessing Romantic-Relationship Quality
Unpacking the Relationship Between Bias-Based Cyberbullying and Internalizing Symptoms Amongst Sexual Minority Youth
Does bureaucracy demotivate public servants? An assessment of psychological mechanisms and the moderating role of age
A connotation to public sector creativity
Growing together
What we mean when we say semantic
I’m not sure that curve means what you think it means
Mindful together
Reliability generalization meta-analysis
Artificial intelligence in scale development
Beyond binary group categorization
Know thyself through data
Introducing Item-Specific Formatting to Scales of Criminal Justice Attitudes
Towards conceptual clarity in pedagogical research
How to measure belonging in higher education
The validation crisis in the L2 motivational self system tradition
Study and instrument quality in perception-based L2 pronunciation research
Measurement Properties of a Standardized Elicited Imitation Test
“I’m so dumb and worthless right now ”
Is Philosophy Exceptional? A Corpus-Based, Quantitative Study
Epistemic Virtue, Cognitive Diversity, and the Division of Labor
Comparing mental health and mental capacity law data across borders
Exploring data use in nonprofit organizations
Behind the Blackpill
Inoculating Against Moral Disengagement Creates Ethical Adherence for Narcissism
The Epidemic of Loneliness
How Do Invested Partners Become Invested? A Prospective Investigation of Fledgling Relationship Development
How to measure interdisciplinary research? A systemic design for the model of measurement
Assessing transparency and methodological precision in variable measurement within organizational research
Education researchers’ beliefs and barriers towards data sharing
The effect of measurement error on clustering
Measuring wellbeing in the Global Flourishing Study
Transparency in (Quasi‐)Experimental L2 Research
The Development and Validation of the Short Form of the Foreign Language Enjoyment Scale
An Update to Methods Showcase Articles in Language Learning
Pragmatic measurement of mechanisms
Research Review
Mapping the Landscape of Pro-Environmental Behavior
Linguistic alignment with an artificial agent
Mapping the self in self‐regulation using complex dynamic systems approach
Improving careers science
Self-perceived creativity and curiosity in middle childhood
Construct jangle or construct mangle? Thinking straight about (nonredundant) psychological constructs
The UBC State Social Connection Scale
Taking communication science and research methodology seriously
Measures of Relationship Power Dynamics in Romantic Relationships
Reflections on the house-of-mirrors
The theory crisis in criminology
An agenda for future research regarding the mental health of young people with care experience
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
Loneliness and the Big Five Personality Traits
Do Data Characteristics Change According to the Number of Scale Points Used? An Experiment Using 5-Point, 7-Point and 10-Point Scales
The 52 symptoms of major depression
Increasing Transparency Through a Multiverse Analysis
What You See Is What You Get? Enhancing Methodological Transparency in Management Research
Many Analysts, One Data Set
False-Positive Psychology
Construct Validation in Social and Personality Research
The jingle and jangle of emotion assessment
Charisma
Much ado about grit
Benefits of open and high-powered research outweigh costs
Context and serial-order effects in personality measurement
The association between adolescent well-being and digital technology use
Construct validity in psychological tests
| Unique citing works | 165 |
|---|---|
| Citations per year | 23,57 |
| Citation span | 2019 - 2026 (8) |
| Citation velocity | current |
| Highly cited | Yes |
| Citation types | Neutral: 146 |