Model-Selection Theory
The Need for a More Nuanced Picture of Use-Novelty and Double-Counting
Bibliographic Data
| ID | 8396335 |
|---|---|
| Authors | Karen Steele (0000-0003-4042-1822, Australian National University, corresponding author), Katie Steele, Charlotte Werndl (0000-0003-2980-8436, London School of Economics and Political Science, corresponding author) |
| Year | 2018 |
| Volume | 69 |
| Issue | 2 |
| Pages | 351-375 |
| Publication date | 2018-06-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | The British Journal for the Philosophy of Science (JOURNAL) |
| Journal identifiers | ISSN: 0007-0882 • E-ISSN: 1464-3537 |
| Publisher | Oxford University Press (PUBLISHER • GB) |
| DOI | 10.1093/bjps/axw024 |
| PMID | 29780170 |
| OpenAlex | W2186355904 |
| Language | EN |
| Citations received | 5 |
| References cited | 20 |
This article argues that common intuitions regarding (a) the specialness of ‘use-novel’ data for confirmation and (b) that this specialness implies the ‘no-double-counting rule’, which says that data used in ‘constructing’ (calibrating) a model cannot also play a role in confirming the model’s predictions, are too crude. The intuitions in question are pertinent in all the sciences, but we appeal to a climate science case study to illustrate what is at stake. Our strategy is to analyse the intuitive claims in light of prominent accounts of confirmation of model predictions. We show that on the Bayesian account of confirmation, and also on the standard classical hypothesis-testing account, claims (a) and (b) are not generally true; but for some select cases, it is possible to distinguish data used for calibration from use-novel data, where only the latter confirm. The more specialized classical model-selection methods, on the other hand, uphold a nuanced version of claim (a), but this comes apart from (b), which must be rejected in favour of a more refined account of the relationship between calibration and confirmation. Thus, depending on the framework of confirmation, either the scope or the simplicity of the intuitive position must be revised. 1 Introduction2 A Climate Case Study3 The Bayesian Method vis-à-vis Intuitions4 Classical Tests vis-à-vis Intuitions5 Classical Model-Selection Methods vis-à-vis Intuitions 5.1 Introducing classical model-selection methods 5.2 Two cases6 Re-examining Our Case Study7 Conclusion
Appeal · Bayesian inference · Bayesian probability · Calibration · Econometrics · Economics · Epistemology · Mathematical economics · Model selection · Novelty · Occam's razor · Political science · Position (finance) · Scope (computer science) · Selection (genetic algorithm) · Simplicity · Statistics · Artificial Intelligence · Bayesian Modeling and Causal Inference · Computer Science · Law · Mathematics · Meta-analysis and systematic reviews · Philosophy · Philosophy and History of Science · Psychology · Social Psychology
Philosophy of Climate Science Part II
A framework for implementing evidence in policymaking
We Have Big Data, But Do We Need Big Theory? Review-Based Remarks on an Emerging Problem in the Social Sciences
Structural uncertainty through the lens of model building
Simplicity and the Sub-Family Problem for Model Selection
Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences
Cross-Validatory Choice and Assessment of Statistical Predictions
A survey of cross-validation procedures for model selection
Some surprising facts about (the problem of) surprising facts
Bayesian pseudo-confirmation, use-novelty, and genuine confirmation
Prediction and accommodation revisited
Model Selection, Simplicity, and Scientific Inference
Novel Evidence and Severe Tests
A Philosopher's Guide to Empirical Success
Logical versus Historical Theories of Confirmation
Prediction Versus Accommodation and the Risk of Overfitting
How to Discount Double-Counting When It Counts
Climate Models, Calibration, and Confirmation
| Unique citing works | 5 |
|---|---|
| Citations per year | 0,45 |
| Citation span | 2015 - 2024 (10) |
| Citation velocity | recent |
| Highly cited | No |
| Citation types | Neutral: 4 |