Knowing how effective an intervention, treatment, or manipulation is and increasing replication rates
Accuracy in parameter estimation as a partial solution to the replication crisis
Bibliographic Data
| ID | 21579693 |
|---|---|
| Authors | Gjalt-Jorn Peters (0000-0002-0336-9589, Open University of the Netherlands, corresponding author), Rik Crutzen (0000-0002-3731-6610, Maastricht University) |
| Year | 2021 |
| Volume | 36 |
| Issue | 1 |
| Pages | 59-77 |
| Publication date | 2021-01-02 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Psychology & Health (JOURNAL) |
| Journal identifiers | ISSN: 0887-0446 • E-ISSN: 1476-8321 |
| Publisher | Informa UK Limited (PUBLISHER • GB) |
| DOI | 10.1080/08870446.2020.1757098 |
| PMID | 32378428 |
| OpenAlex | W2915081097 |
| Language | EN |
| Citations received | 4 |
| References cited | 49 |
OBJECTIVE: Although basing conclusions on confidence intervals for effect size estimates is preferred over relying on null hypothesis significance testing alone, confidence intervals in psychology are typically very wide. One reason may be a lack of easily applicable methods for planning studies to achieve sufficiently tight confidence intervals. This paper presents tables and freely accessible tools to facilitate planning studies for the desired accuracy in parameter estimation for a common effect size (Cohen's d). In addition, the importance of such accuracy is demonstrated using data from the Reproducibility Project: Psychology (RPP). RESULTS: It is shown that the sampling distribution of Cohen's d is very wide unless sample sizes are considerably larger than what is common in psychology studies. This means that effect size estimates can vary substantially from sample to sample, even with perfect replications. The RPP replications' confidence intervals for Cohen's d have widths of around 1 standard deviation (95% confidence interval from 1.05 to 1.39). Therefore, point estimates obtained in replications are likely to vary substantially from the estimates from earlier studies. CONCLUSION: The implication is that researchers in psychology -and funders- will have to get used to conducting considerably larger studies if they are to build a strong evidence base
Economics · Estimation · Advanced Causal Inference Techniques · Computer Science · Medicine · Meta-analysis and systematic reviews · Psychometric Methodologies and Testing · Virology
Relevant determinants of Dutch patients’ informed decision-making and use of online access to medical records
Social-Cognitive Determinants of HIV Testing Among Tuberculosis Infected Patients in Kassala State, Sudan
Identification of Relevant Sociocognitive Determinants Explaining Multiple Parental Sun Protection Behaviors
Meaningful change definitions
The Rules of the Game Called Psychological Science
Scientists rise up against statistical significance
Sailing From the Seas of Chaos Into the Corridor of Stability
Moving to a World Beyond “ p < 0.05”
Replication, falsification, and the crisis of confidence in social psychology
Confidence intervals for population reliability coefficients
The ASA Statement on p -Values
Sample Size Planning for Statistical Power and Accuracy in Parameter Estimation
Sample Size Requirements for Estimating Pearson, Kendall and Spearman Correlations
Confidence intervals rather than P values
The fallacy of placing confidence in confidence intervals
A taxonomy of behaviour change methods
Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature
A Multilab Preregistered Replication of the Ego-Depletion Effect
Statistical tests, P values, confidence intervals, and power
Calculating and reporting effect sizes to facilitate cumulative science
G*Power 3
Estimating the reproducibility of psychological science
The New Statistics
Equivalence Tests
A Primer on the Understanding, Use, and Calculation of Confidence Intervals that are Based on Central and Noncentral Distributions
The Precision of Effect Size Estimation From Published Psychological Research
At what sample size do correlations stabilize
What Future Quantitative Social Science Research Could Look Like
Things I have learned (so far)
Publication Manual of the American Psychological Association
A taxonomy of behavior change techniques used in interventions
A power primer
| Citations per year | 0.8 |
|---|---|
| Citation span | 2021 - 2025 (5) |
| Highly cited | No |