How Much Should We Trust Instrumental Variable Estimates in Political Science? Practical Advice Based on 67 Replicated Studies
Bibliographic Data
| ID | 7971079 |
|---|---|
| Authors | Apoorva Lal (0000-0002-3697-614X), Mackenzie Lockhart (0000-0003-2861-0698, Yale University), Yiqing Xu (0000-0003-2041-6671, Stanford University, corresponding author), Ziwen Zu (0000-0002-4852-4415, University of California San Diego) |
| Year | 2024 |
| Volume | 32 |
| Issue | 4 |
| Pages | 521-540 |
| Publication date | 2024-10-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Political Analysis (JOURNAL) |
| Journal identifiers | ISSN: 1047-1987 • E-ISSN: 1476-4989 |
| Publisher | Cambridge University Press (CUP) (PUBLISHER) |
| DOI | 10.1017/pan.2024.2 |
| OpenAlex | W4396617588 |
| Language | EN |
| Citations received | 24 |
| References cited | 65 |
Instrumental variable (IV) strategies are widely used in political science to establish causal relationships, but the identifying assumptions required by an IV design are demanding, and assessing their validity remains challenging. In this paper, we replicate 67 articles published in three top political science journals from 2010 to 2022 and identify several concerning patterns. First, researchers often overestimate the strength of their instruments due to non-i.i.d. error structures such as clustering. Second, IV estimates are often highly uncertain, and the commonly used t -test for two-stage-least-squares (2SLS) estimates frequently underestimate the uncertainties. Third, in most replicated studies, 2SLS estimates are significantly larger in magnitude than ordinary-least-squares estimates, and their absolute ratio is inversely related to the strength of the instrument in observational studies—a pattern not observed in experimental ones—suggesting potential violations of unconfoundedness or the exclusion restriction in the former. We provide a checklist and software to help researchers avoid these pitfalls and improve their practice
Advice (programming · Data science · Econometrics · Economics · Instrumental variable · Political science · Politics · Variable (mathematics · Advanced Causal Inference Techniques · Computer Science · Electoral Systems and Political Participation · Law · Mathematics · Psychology · Qualitative Comparative Analysis Research
Sharpening Blunt Instruments
Moral Force
From field trials to farm adoption
The business case for demographic diversity in strategic leadership teams
Rural electrification, the credibility revolution, and the limits of evidence-based policy
The Impact of a Preschool Childcare Expansion on Grade Progression in Early School Careers
The institutional impact of left-leaning populism in Latin America
The IMF, Structural Adjustment, and Poverty
For God, Tsar and fatherland? The church’s role in modern autocracy
Beyond the haze
Blaming the wind? The impact of wind turbine on bird biodiversity
Beyond Psychological Costs
How nuclear power hurts the Greens
Weapons and influence
Workplace Peer Effects in Turnout
Attitudinal and Behavioral Legacies of Wartime Violence
Handle with Care
Rain, rain, go away
The Development of an Issue Public
The Technological Bridge
Aspiring to Success
Does Affluence Influence Authoritarian Responsiveness? Evidence From Urban China
The Backlash Against Civil Society Participation in International Organizations
Causal Panel Analysis under Parallel Trends
Valid t -ratio Inference for IV
Weak Instruments in Instrumental Variables Regression
Semiparametric instrumental variable estimation of treatment response models
Beyond Power Calculations
When Should You Adjust Standard Errors for Clustering?
Estimation of the Parameters of a Single Equation in a Complete System of Stochastic Equations
A Conditional Likelihood Ratio Test for Structural Models
Asymptotic Distributions of Instrumental Variables Statistics with Many Instruments
The weak instrument problem of the system GMM estimator in dynamic panel data models
A Practitioner’s Guide to Cluster-Robust Inference
Identification of Causal Effects Using Instrumental Variables
Large Sample Properties of Generalized Method of Moments Estimators
Instrumental Variables Regression with Weak Instruments
One instrument to rule them all
Mostly Harmless Econometrics
Sargan's Intrumental Variables Estimation and the Generalized Method of Moments
A Robust Test for Weak Instruments
A Survey of Weak Instruments and Weak Identification in Generalized Method of Moments
Plausibly Exogenous
Bootstrap-Based Improvements for Inference with Clustered Errors
Empirical Decomposition of the IV-OLS Gap with Heterogeneous and Nonlinear Effects
On the reliability of published findings using the regression discontinuity design in political science
How Are We Doing? Data Access and Replication in Political Science
Profiling Compliers and Noncompliers for Instrumental-Variable Analysis
Instrumental Variables Estimation in Political Science
Bases, Bullets, and Ballots
Undermining Authoritarian Innovation
Mediating the Electoral Connection
How Large Are the Political Costs of Fiscal Austerity
Credible Sources and Sophisticated Voters
Practical and Effective Approaches to Dealing With Clustered Data
How Much Should We Trust Estimates from Multiplicative Interaction Models? Simple Tools to Improve Empirical Practice
On the Use and Abuse of Spatial Instruments
Small Aggregates, Big Manipulation
Direct Democracy and Women's Political Engagement
Does Choice Bring Loyalty? Electoral Participation and the Development of Party Identification
MNCs, Rents, and Corruption
Inclusion and Public Policy
Do Inheritance Customs Affect Political and Social Inequality
When Natural Experiments Are Neither Natural nor Experiments
Gone For Good
Collective Action and Representation in Autocracies
Democratization and the Conditional Dynamics of Income Distribution
| Unique citing works | 24 |
|---|---|
| Citations per year | 4 |
| Citation span | 2020 - 2026 (7) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 24 |