Research news and Comment

Reflections on Statistical and Substantive Significance, With a Slice of Replication

Dados Bibliográficos

ID9734619
AutoresDaniel H Robinson (0000-0003-0384-704X, autor correspondente), Joel R Levin
Ano1997
Volume26
Fascículo5
Páginas21-26
Data de publicação1997-06-01
Peer ReviewedSim
Open AccessSim
TipoARTICLE
PeriódicoEducational Researcher (JOURNAL)
Identificadores do periódicoISSN: 0013-189X • E-ISSN: 1935-102X
EditoraAmerican Educational Research Association (AERA) (PUBLISHER)
DOI10.3102/0013189x026005021
OpenAlexW1963500756
IdiomaEN
Citações recebidas24
Referências citadas13

In this comment, we propose some modifications to Thompson’s (1996) recent suggestions for AERA editorial policy on statistical significance testing. First, we discuss the potential problems, both procedural and conceptual, that could arise from professional journals mandating the addition of the modifier “statistically” to “significant.” If language remains an issue of consequence, however, findings stemming from rejected null hypotheses could be termed “nonchance” to communicate simply and more precisely what it is that a statistical test reveals. Second, although asking authors to provide explicit effect-size information is both sensible and useful, we illustrate how effect sizes (like p values) can be misinterpreted and misused. Finally, in accord with Thompson, we argue that greater attention to replication should be encouraged in educational research. At the same time, we believe that internal replication analyses do not represent adequate substitutes for (or even close approximations to) external replication studies based on new, independent participants. With journals welcoming both constructed and literal replication studies, consumers of the educational research literature can concentrate less on the limited believability of one-time findings and more on the enhanced believability of repeatable and generalizable ones

Cognitive psychology · Econometrics · Epistemology · Null (SQL · Null hypothesis · Replication (statistics · Statistical hypothesis testing · Statistics · Test (biology · Advanced Causal Inference Techniques · Computer Science · Educational Assessment and Improvement · Mathematics · Meta-analysis and systematic reviews · Psychology · Social Psychology