Enhancing Validity in Observational Settings When Replication is Not Possible
Bibliographic Data
| ID | 6342001 |
|---|---|
| Authors | Christopher J Fari (0000-0001-9837-186X), Zachary M Jones (0000-0001-6423-957X), Zachary Jones (0000-0002-7523-0471) |
| Year | 2018 |
| Volume | 6 |
| Issue | 2 |
| Pages | 365-380 |
| Publication date | 2018-04-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Political Science Research and Methods (JOURNAL) |
| Journal identifiers | ISSN: 2049-8470 • E-ISSN: 2049-8489 |
| Publisher | Cambridge University Press (CUP) (PUBLISHER) |
| DOI | 10.1017/psrm.2017.5 |
| OpenAlex | W3122927233 |
| Language | EN |
| Citations received | 8 |
| References cited | 59 |
We argue that political sciexntists can provide additional evidence for the predictive validity of observational and quasi-experimental research designs by minimizing the expected prediction error or generalization error of their empirical models. For observational and quasi-experimental data not generated by a stochastic mechanism under the researcher’s control, the reproduction of statistical analyses is possible but replication of the data-generating procedures is not. Estimating the generalization error of a model for this type of data and then adjusting the model to minimize this estimate—regularization—provides evidence for the predictive validity of the study by decreasing the risk of overfitting. Estimating generalization error also allows for model comparisons that highlight underfitting: when a model generalizes poorly due to missing systematic features of the data-generating process. Thus, minimizing generalization error provides a principled method for modeling relationships between variables that are measured but whose relationships with the outcome(s) are left unspecified by a deductively valid theory. Overall, the minimization of generalization error is important because it quantifies the expected reliability of predictions in a way that is similar to external validity, consequently increasing the validity of the study’s conclusions
Artificial neural network · Econometrics · External validity · Generalization · Generalization error · Machine learning · Observational study · Overfitting · Regularization (linguistics · Reliability (semiconductor · Replication (statistics · Statistics · Advanced Causal Inference Techniques · Computer Science · Health Systems, Economic Evaluations, Quality of Life · Mathematics · Qualitative Comparative Analysis Research · Artificial Intelligence
Evaluating explanations for poverty selectivity in foreign aid
Forecasting Peace Agreement Content
The role of hyperparameters in machine learning models and how to tune them
Territorial Contenders in World Politics
Reassessing the Role of Theory and Machine Learning in Forecasting Civil Conflict
External Validity and Meta‐Analysis
Ideology Justifies Morality
Innovations in concepts and measurement for the study of peace and conflict
Semiparametric Regression for the Social Sciences
The Jackknife, the Bootstrap and Other Resampling Plans
An Introduction to the Bootstrap
Regularization and Variable Selection Via the Elastic Net
Philosophy and the practice of Bayesian statistics
The Bayesian Lasso
Least angle regression
Bilinear Mixed-Effects Models for Dyadic Data
Model-Based Clustering for Social Networks
To Explain or to Predict?
Estimation and Inference of Heterogeneous Treatment Effects using Random Forests
Latent Space Approaches to Social Network Analysis
Unbiased Recursive Partitioning
A survey of cross-validation procedures for model selection
Greedy function approximation
Regression Shrinkage and Selection Via the Lasso
Natural Experiments in the Social Sciences
Ensemble forecasting of irregular leadership change
Some Like It Hot
Natural Experiments Based on Geography
Regression Analysis
Dynamic Patterns of Human Rights Practices
The Bar Fight Theory of International Conflict
The Statistics of Causal Inference
Fightin' Words
Kernel Regularized Least Squares
Can Structural Conditions Explain the Onset of Nonviolent Uprisings
War Is in the Error Term
Comparable Preference Estimates across Time and Institutions for the Court, Congress, and Presidency
Causal Heterogeneity in Comparative Research
Beyond Linearity by Default
How to Analyze Political Attention with Minimal Assumptions and Costs
The Changing Standard of Accountability and the Positive Relationship between Human Rights Treaty Ratification and Compliance
Publication, Publication
Replication, Verification, Secondary Analysis, and Data Collection in Political Science
Science Deserves Better
Theory is dead, long live theory
The perils of policy by p-value
Measurement Validity
Enhancing the Validity and Cross-Cultural Comparability of Measurement in Survey Research
An Empirical Evaluation of Explanations for State Repression
Respect for Human Rights has Improved Over Time
| Unique citing works | 8 |
|---|---|
| Citations per year | 1,14 |
| Citation span | 2019 - 2026 (8) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 8 |