The potential of benchmark challenges in the social sciences
Bibliographic Data
| ID | 3079735 |
|---|---|
| Authors | Paulina Pankowska (0000-0001-6226-6814, Utrecht University, the Netherlands, corresponding author), Adrienne Mendrik (0000-0001-6631-7068, Eyra, The Netherlands), Tepora Emery (0000-0001-6137-9577, ODISSEI & Erasmus University Rotterdam, The Netherlands), Javier Garcia-Bernardo (Utrecht University, the Netherlands), Javier Garcia‐bernardo (0000-0002-6119-1790, Utrecht University) |
| Year | 2024 |
| Volume | 63 |
| Issue | 4 |
| Pages | 498-519 |
| Publication date | 2024-12-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Social Science Information (JOURNAL) |
| Journal identifiers | ISSN: 0539-0184 • E-ISSN: 1461-7412 |
| Publisher | SAGE Publications Inc (PUBLISHER) |
| DOI | 10.1177/05390184241297742 |
| OpenAlex | W4404859489 |
| Language | EN |
| Citations received | 2 |
| References cited | 31 |
Social scientists aim to create explanations of the world. For each social phenomenon, scientists have proposed a myriad of theories to explain its working mechanisms. Traditionally, these theories are tested by generating hypotheses, translating them into a statistical model, and assessing the significance of the model's coefficients. Such an approach, however, often leads to the specification of a large number of (at times contradictory) models, all asserting that they capture the same theory. As things currently stand, there is no framework that allows for a comparison of these models. In this article, we argue that benchmarks can serve as a standard frame of reference that can help to determine which models fit better with empirical observations in a specific context. A benchmark is a standardized validation framework that allows for a direct comparison of the prediction accuracy of various models that address the same research problem. We outline the potential of organizing benchmark challenges in the social sciences and provide recommendations for their utilization
Data science · Empirical research · Management science · Statistics · Computer Science · Ecosystem dynamics and resilience · Innovation, Sustainability, Human-Machine Systems · Mathematics · Species Distribution and Climate Change · Artificial Intelligence
Integrating explanation and prediction in computational social science
ImageNet Large Scale Visual Recognition Challenge
Prediction and explanation in social systems
The Fair Guiding Principles for scientific data management and stewardship
A Pragmatist’s Guide to Using Prediction in the Social Sciences
Fragile Families
Should social science be more solution-oriented
The potential of benchmark challenges in the social sciences
Introduction to the Special Collection on the Fragile Families Challenge
Successes and Struggles with Computational Reproducibility
Imputing Data for the Fragile Families Challenge
Improving Metadata Infrastructure for Complex Surveys
Privacy, Ethics, and Data Access
| Unique citing works | 2 |
|---|---|
| Citations per year | 1 |
| Citation span | 2024 - 2026 (3) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 2 |