Meta-Analysis as a Research Synthesis Methodology
Cause for Concern
Bibliographic Data
| ID | 11214466 |
|---|---|
| Authors | William R Nugent (0000-0003-0811-3221, University of Tennessee at Knoxville, corresponding author) |
| Year | 2009 |
| Volume | 35 |
| Issue | 2 |
| Pages | 181-192 |
| Publication date | 2009-03-04 |
| Open Access | No |
| Type | ARTICLE |
| Venue | Journal of Social Service Research (JOURNAL) |
| Journal identifiers | ISSN: 0148-8376 • E-ISSN: 1540-7314 |
| Publisher | Informa UK Limited (PUBLISHER • GB) |
| DOI | 10.1080/01488370802678967 |
| OpenAlex | W2043403044 |
| Language | EN |
| Citations received | 3 |
| References cited | 9 |
The use of meta-analysis is growing in popularity. It is based on the fundamental notion of the effect size, and a critical assumption is that effect sizes based on different measures are directly comparable. In this article it is argued that the direct comparability of effect sizes across measures implies the invariance of the effect sizes across these measures. A model of standardized mean difference (SMD) effect size invariance is developed, based on multifacet generalizability theory, which shows that SMD effect size invariance requires certain validity invariance conditions to hold. One implication of these findings is that the direct comparability of SMD effect sizes based on different measurement procedures is an empirical matter requiring testing prior to conducting a meta-analysis. Findings are also discussed suggesting that violations of one of these conditions—universe score validity invariance—can bring about substantial differences across SMD effect sizes as a function of measurement procedure. These findings suggest the need for a more refined use of meta-analysis since meta-analytic results may be adversely impacted by the lack of direct comparability of effect sizes based on different measures
Comparability · Confirmatory factor analysis · Econometrics · Generalizability theory · Measurement invariance · Meta-analysis · Popularity · Sample size determination · Statistics · Strictly standardized mean difference · Structural equation modeling · Mathematics · Medicine · Meta-analysis and systematic reviews · Psychology · Social Psychology
Statistical Theories of Mental Test Scores
Generalizability Theory
Construct Validity Invariance and Discrepancies in Meta-Analytic Effect Sizes Based on Different Measures
The Comparability of the Standardized Mean Difference Effect Size Across Different Measures of the Same Construct
The efficacy of psychological, educational, and behavioral treatment
| Citations per year | 0.2 |
|---|---|
| Citation span | 2011 - 2011 (1) |
| Highly cited | No |