Just an artifact? The concordance between peer review and bibliometrics in economics and statistics in the Italian research assessment exercise
Bibliographic Data
| ID | 21255812 |
|---|---|
| Authors | Alberto Baccini (0000-0003-0293-482X, University of Siena), Giuseppe De Nicolao (0000-0002-3712-9911, University of Pavia) |
| Year | 2022 |
| Volume | 3 |
| Issue | 1 |
| Pages | 194-207 |
| Publication date | 2022-04-12 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Quantitative Science Studies (JOURNAL) |
| Journal identifiers | ISSN: 2641-3337 • E-ISSN: 2641-3337 |
| Publisher | MIT Press (PUBLISHER • US) |
| DOI | 10.1162/qss_a_00172 |
| OpenAlex | W4200043188 |
| Language | EN |
| Citations received | 2 |
| References cited | 18 |
During the Italian research assessment exercise (2004–2010), the governmental agency (ANVUR) in charge of its realization performed an experiment on the concordance between peer review and bibliometrics at an individual article level. The computed concordances were at most weak for science, technology, engineering, and mathematics. The only exception was the moderate concordance found for the area of economics and statistics. In this paper, the disclosed raw data of the experiment are used to shed light on the anomalous results obtained for economics and statistics. In particular, the data permit us to document that the protocol of the experiment adopted for economics and statistics was different from the one used in the other areas. Indeed, in economics and statistics the same group of scholars developed the bibliometric ranking of journals for evaluating articles, managing peer reviews and forming the consensus groups for deciding the final scores of articles after having received the referee’s reports. This paper shows that the highest level of concordance in economics and statistics was an artifact mainly due to the role played by consensus groups in boosting the agreement between bibliometrics and peer review
Actuarial science · Bibliometrics · Concordance · Data science · Economics · Information retrieval · Library science · Social science · Sociology · Statistics · Computer Science · Mathematics · Medicine · scientometrics and bibliometrics research
| Citations per year | 2 |
|---|---|
| Citation span | 2025 - 2025 (1) |
| Highly cited | No |