Why many studies of individual differences with inhibition tasks may not localize correlations
Bibliographic Data
| ID | 21640295 |
|---|---|
| Authors | Jeffrey N Rouder (0000-0003-2023-3891, University of California, Irvine, corresponding author), Aakriti Kumar (0000-0002-9502-013X, University of California, Irvine), Julia M Haaf (0000-0001-5122-706X, Amsterdam University of the Arts) |
| Year | 2023 |
| Volume | 30 |
| Issue | 6 |
| Pages | 2049-2066 |
| Publication date | 2023-12-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Psychonomic Bulletin & Review (JOURNAL) |
| Journal identifiers | ISSN: 1069-9384 • E-ISSN: 1531-5320 |
| Publisher | Springer Science and Business Media LLC (PUBLISHER) |
| DOI | 10.3758/s13423-023-02293-3 |
| PMID | 37450264 |
| OpenAlex | W4384301245 |
| Language | EN |
| Citations received | 12 |
| References cited | 49 |
Individual difference exploration of cognitive domains is predicated on being able to ascertain how well performance on tasks covary. Yet, establishing correlations among common inhibition tasks such as Stroop or flanker tasks has proven quite difficult. It remains unclear whether this difficulty occurs because there truly is a lack of correlation or whether analytic techniques to localize correlations perform poorly real-world contexts because of excessive measurement error from trial noise. In this paper, we explore how well correlations may localized in large data sets with many people, tasks, and replicate trials. Using hierarchical models to separate trial noise from true individual variability, we show that trial noise in 24 extant tasks is about 8 times greater than individual variability. This degree of trial noise results in massive attenuation in correlations and instability in Spearman corrections. We then develop hierarchical models that account for variation across trials, variation across individuals, and covariation across individuals and tasks. These hierarchical models also perform poorly in localizing correlations. The advantage of these models is not in estimation efficiency, but in providing a sense of uncertainty so that researchers are less likely to misinterpret variability in their data. We discuss possible improvements to study designs to help localize correlations
Cognition · Cognitive psychology · Correlation · Extant taxon · Machine learning · Multilevel model · Replicate · Statistics · Stroop effect · Computer Science · Decision-Making and Behavioral Economics · Neural and Behavioral Psychology Studies · Neural dynamics and brain function · Neuroscience · Psychology · Artificial Intelligence
Mapping the Relationship Between Core Executive Functions and Mind Wandering in Children and Adolescents
Visual distraction, inhibitory control, and children’s math performance
Ten years Diffusion Model for Conflict (DMC) tasks
Neither measurement error nor speed–accuracy trade-offs explain the difficulty of establishing attentional control as a psychometric construct
Individual differences do not mask effects of unconscious processing
Random-effects psychophysics for studying individual differences in perception and cognition
Comparing the reliability of individual differences for various measurement models in conflict tasks
A qualitative systematic review of individual differences in Stroop task performance among healthy adults
Group level and individual differences in second language sentence processing
Measurement and sampling noise undermine inferences about awareness in location probability learning
Replicate Me if You Can
Precise individual measures of inhibitory control
Generalized Latent Variable Modeling
Statistical Rethinking
A psychometrics of individual differences in experimental tasks
No evidence for bilingual cognitive advantages
Working-memory capacity and the control of attention
A controlled-attention view of working-memory capacity.
The Proof and Measurement of Association between Two Things
Effects of noise letters upon the identification of a target letter in a nonsearch task
Generating random correlation matrices based on vines and extended onion method
Individual differences in working memory and reading
Large-scale analysis of test–retest reliabilities of self-regulation measures
On the ability to inhibit thought and action
A consensus guide to capturing the ability to inhibit actions and impulsive behaviors in the stop-signal task
Blavaan
Structural Equations with Latent Variables
Cognitive efficiency beats top-down control as a reliable individual difference dimension relevant to self-control
Half a century of research on the Stroop effect
Reaction time in differential and developmental research
The reliability paradox
A Six-Factor Structure of Personality-Descriptive Adjectives
Personality trait structure as a human universal
The two disciplines of scientific psychology
The Unity and Diversity of Executive Functions and Their Contributions to Complex “Frontal Lobe” Tasks
| Unique citing works | 12 |
|---|---|
| Citations per year | 12 |
| Citation span | 2025 - 2026 (2) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 12 |