How to embrace variation and accept uncertainty in linguistic and psycholinguistic data analysis

Bibliographic Data

ID7913367
AuthorsShravan Vasishth (0000-0003-2027-1994, University of Potsdam, corresponding author), Andrew Gelman (0000-0002-6975-2601, Columbia University)
Year2021
Volume59
Issue5
Pages1311-1342
Publication date2021-09-27
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueLinguistics (JOURNAL)
Journal identifiersISSN: 0024-3949 • E-ISSN: 1613-396X
PublisherWalter de Gruyter GmbH (PUBLISHER • DE)
DOI10.1515/ling-2019-0051
OpenAlexW3165824770
LanguageEN
Citations received23
References cited67

The use of statistical inference in linguistics and related areas like psychology typically involves a binary decision: either reject or accept some null hypothesis using statistical significance testing. When statistical power is low, this frequentist data-analytic approach breaks down: null results are uninformative, and effect size estimates associated with significant results are overestimated. Using an example from psycholinguistics, several alternative approaches are demonstrated for reporting inconsistencies between the data and a theoretical prediction. The key here is to focus on committing to a falsifiable prediction, on quantifying uncertainty statistically, and learning to accept the fact that – in almost all practical data analysis situations – we can only draw uncertain conclusions from data, regardless of whether we manage to obtain statistical significance or not. A focus on uncertainty quantification is likely to lead to fewer excessively bold claims that, on closer investigation, may turn out to be not supported by the data

Bayesian inference · Bayesian probability · Cognition · Cognitive psychology · Data mining · Econometrics · Focus (optics) · Frequentist inference · Frequentist probability · Inference · Linguistics · Null (SQL) · Null hypothesis · Psycholinguistics · Statistical hypothesis testing · Statistical inference · Statistical power · Statistics · Variation (astronomy) · Artificial Intelligence · Computer Science · Mathematics · Natural Language Processing Techniques · Neurobiology of Language and Bilingualism · Psychology · Syntax, Semantics, Linguistic Variation

Tools

Open DOIOpen Access