The Power of Bias in Economics Research
Bibliographic Data
| ID | 9720765 |
|---|---|
| Authors | John P A Ioannidis (0000-0003-3118-6859, Stanford University), T D Stanley (0000-0002-3205-1983, Hendrix College), Hristos Doucouliagos (0000-0001-5269-3556, Deakin University) |
| Year | 2017 |
| Volume | 127 |
| Issue | 605 |
| Pages | F236-F265 |
| Publication date | 2017-10-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | The Economic Journal (JOURNAL) |
| Journal identifiers | ISSN: 0013-0133 • E-ISSN: 1468-0297 |
| Publisher | Oxford University Press (OUP) (PUBLISHER) |
| DOI | 10.1111/ecoj.12461 |
| OpenAlex | W2765949027 |
| Language | EN |
| Citations received | 100 |
| References cited | 53 |
We investigate two critical dimensions of the credibility of empirical economics research: statistical power and bias. We survey 159 empirical economics literatures that draw upon 64,076 estimates of economic parameters reported in more than 6,700 empirical studies. Half of the research areas have nearly 90% of their results under‐powered. The median statistical power is 18%, or less. A simple weighted average of those reported results that are adequately powered (power ≥ 80%) reveals that nearly 80% of the reported effects in these empirical economics literatures are exaggerated; typically, by a factor of two and with one‐third inflated by a factor of four or more
Credibility · Econometrics · Economics · Empirical research · Physics · Political science · Positive economics · Power (physics · Statistical power · Statistics · Thermodynamics · Decision-Making and Behavioral Economics · Experimental Behavioral Economics Studies · Forecasting Techniques and Applications · Law · Mathematics
Can community policing improve police-community relations in a low-income country setting?
What meta-analyses reveal about the replicability of psychological research
The Production of Knowledge
Internal Conflicts and Shocks. A Narrative Meta-Analysis
Aid, China, and Growth
Transparency, Reproducibility, and the Credibility of Economics Research
Correcting for Bias in Psychology
Exploratory Research
Early Childhood Education and Care and mother’s labor supply
The lead-crime hypothesis
The impact of large-scale agricultural investments on welfare of local communities
Does paid parental leave affect children's schooling outcomes? Replicating Danzer and Lavy (2018)
How often do dictators have positive economic effects? Global evidence, 1858–2010
On the contribution of experiments to an ontology of economics
Dude, Where’s My Treatment Effect? Errors in Administrative Data Linking and the Destruction of Statistical Power in Randomized Experiments
Does economic globalisation promote economic growth? A meta‐analysis
The effect of health on economic growth
Institutions and FDI from BRICS countries
Quest for the general effect size of finance on growth
A meta-analysis on the effect of technology on the achievement of less advantaged students
Work Hard or Play Hard? Degree Class, Student Leadership and Employment Opportunities
Does the Left Spend More? An Econometric Survey of Partisan Politics
Regulating Markets with Advice
Student internships and employment opportunities after graduation
Student Employment and Education
The hierarchy in economics and its implications
Biases in fiscal multiplier estimates
Bias and careers
Income, growth, and democracy looking for the main causal directions in the nexus
Opportunistic privatization
Does democracy cause growth? A meta-analysis (of 2000 regressions)
Rewarding good political behavior
Toward collaborative cost-inclusive evaluation
Patterns of dissertation dissemination
Reproducibility issues with correlating Beall-listed publications and research awards at a small Canadian business school
Toward a More Powerful Experimental Communication Science
Still in Mortal Peril? Recent Research Suggests a New Agenda for Health Care Reform
Inflated effect sizes and underpowered tests
The Returns to Education
Meta‐mining
Up to No Good? Gender, Social Impact Work, and Employee Promotions
All You Need is Cable TV
Publication and Attenuation Biases in Measuring Skill Substitution
Examining Selection Pressures in the Publication Process through the Lens of Sniff Tests
NGOs and the Effectiveness of Interventions
New Evidence and Design Considerations for Repeated Measure Experiments in Survey Research
Do Racial Differences Exist in Survey Data Quality
MeToo…or not? Do salient shocks affect gender social norms
A Systematic Review on the Evolution of Power Analysis Practices in Psychological Research
How to dampen the surge of non-communicable diseases in Southeast Asia
Tool-Driven Revolutions in Archaeological Science
Randomized controlled trials
Harnessing citizen investment in community-based energy initiatives
Promoting Reproducibility and Replicability in Political Science
The ‘intention to stay’ of the floating migrant population
Causal Inference and Impact Evaluation
Preface – Times Have Changed
Economic Growth and Household Purchasing Power in France
Income and Wealth Inequality in France
Forty Years of Change in Labour Supply and Demand by Skill Level – Technical Progress, Labour Costs and Social Change
Fifty Years of Abstracts in the Journal Economie et Statistique
The Emergence and Consolidation of Microsimulation Methods in France
What Value Do We Attach to Climate Action
Labour Share Developments in OECD Countries Over the Past Two Decades
What do we know about the impact of microfinance? The problems of statistical power and precision
Revisiting the Income Inequality-Crime Puzzle
A meta-analysis of the effects of remittances on household education expenditure
Remittances and economic growth
Has Chinese aid benefited recipient countries? Evidence from a meta-regression analysis
Direct vs indirect management training
Timing matters
Do index insurance programs live up to their promises? Aggregating evidence from multiple experiments
How do policymakers update their beliefs
Trials and tribulations
A meta-analysis of lost-letter field experiments
Preregistration and reproducibility
Sex differences in trust and trustworthiness
Behavioral symmetry, rent seeking, and the Republic of Science
How to trust a scientist
Internal conflicts and shocks
P -Hacking, Data type and Data-Sharing Policy
Why science’s crisis should not become a political battling ground
The Black–white lifetime earnings gap
The Research Reproducibility Crisis and Economics of Science
Persuasion Bias in Science
To Replicate or Not to Replicate? Exploring Reproducibility in Economics through the Lens of a Model and a Pilot Study
P -hacking and Significance Stars
What can mathematical modelling contribute to a sociology of quantification
Terrorism and early childhood health outcomes
Transparency and quality assessment of research data in post-Soviet area studies
Achieving Statistical Significance with Control Variables and Without Transparency
Making a promise increases the moral cost of lying
Quantitative Political Science Research Is Greatly Underpowered
Does schooling promote democracy? A meta-analysis
Presidential Unilateral Power
The impact of healthcare spending on health outcomes
Meta-analysis, military expenditures and growth
What Can We Learn about the Effects of Democracy Using Cross-National Data
The path of economics research production
The Wages of Mothers' Labor
Statistical Power Analysis for the Behavioral Sciences
Methods of Meta-Analysis
The Rules of the Game Called Psychological Science
Meta‐Regression Methods for Detecting and Estimating Empirical Effects in the Presence of Publication Selection
Meta‐analysis of Economics Research Reporting Guidelines
How to Make More Published Research True
Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
The Persistence of Underpowered Studies in Psychological Research
P-curve
Meta-regression approximations to reduce publication selection bias
Raise standards for preclinical cancer research
Six Randomized Evaluations of Microcredit
Why Science Is Not Necessarily Self-Correcting
Price and Income Elasticities of Residential Water Demand
Meta‐regression Analysis
Power failure
Beyond Publication Bias
Publication Selection Bias in Minimum‐Wage Research? A Meta‐Regression Analysis
Promoting an open research culture
Bias in meta-analysis detected by a simple, graphical test
Statistical Power Analysis
Reporting the Fragility of Regression Estimates
Data Mining
Are all Economic Hypotheses False
To Replicate or Not to Replicate? Exploring Reproducibility in Economics through the Lens of a Model and a Pilot Study
Antifragile
The file drawer problem and tolerance for null results
| Unique citing works | 100 |
|---|---|
| Citations per year | 11,11 |
| Citation span | 2017 - 2026 (10) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 98 |