Research news and Comment
Statistical Significance is not a “Kosher Certificate” for Observed Effects: A Critical Analysis of the Two-Step Approach to the Evaluation of Empirical Results
Bibliographic Data
| ID | 9733189 |
|---|---|
| Authors | Sorel Cahan (corresponding author) |
| Year | 2000 |
| Volume | 29 |
| Issue | 1 |
| Pages | 31-34 |
| Publication date | 2000-01-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Educational Researcher (JOURNAL) |
| Journal identifiers | ISSN: 0013-189X • E-ISSN: 1935-102X |
| Publisher | American Educational Research Association (AERA) (PUBLISHER) |
| DOI | 10.3102/0013189x029001031 |
| OpenAlex | W2072103227 |
| Language | EN |
| Citations received | 2 |
| References cited | 10 |
In order to prevent misleading conclusions based on spurious observed effects—especially seductively large ones—Robinson and Levin (1997) suggested a two step approach to the reporting and evaluation of empirical results. According to the two-step model, the evaluation of the magnitude and substantive significance of obtained effects should be conditional upon their statistical significance: Author should first indicate whether the observed effect is statistically improbable, and only if it is should they then indicate how large or important it is. The purpose of their paper is (a) to show that even through the underlying intention—to prevent unwarranted evaluation of spurious observed effects—is a laudable one, the two-step approach is inappropriate for this purpose, and (b) to reiterate the preferred approach, namely increased sample size and computation of confidence intervals
Algorithm · Certificate · Confidence interval · Econometrics · Empirical research · Sample (material · Sample size determination · Spurious relationship · Statistical analysis · Statistical significance · Statistics · Computer Science · Data Analysis with R · Mathematics · Meta-analysis and systematic reviews · Psychology · Statistical Methods in Clinical Trials
| Unique citing works | 2 |
|---|---|
| Citations per year | 0,08 |
| Citation span | 2000 - 2003 (4) |
| Citation velocity | historical |
| Highly cited | No |
| Citation types | Neutral: 2 |