Are disruption index indicators convergently valid? The comparison of several indicator variants with assessments by peers

Bibliographic Data

ID21255780
AuthorsLutz Bornmann (0000-0003-0810-7091, Max Planck Society), Sitaram Devarakonda (0000-0001-7722-756X, NETE (United States)), Alexander Tekles (0000-0001-8765-9331, Max Planck Society), George Chacko (0000-0002-2127-1892, NETE (United States))
Year2020
Volume1
Issue3
Pages1242-1259
Publication date2020-08-01
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueQuantitative Science Studies (JOURNAL)
Journal identifiersISSN: 2641-3337 • E-ISSN: 2641-3337
PublisherMIT Press (PUBLISHER • US)
DOI10.1162/qss_a_00068
OpenAlexW2990160624
LanguageEN
Citations received22
References cited43

Recently, Wu, Wang, and Evans (2019) proposed a new family of indicators, which measure whether a scientific publication is disruptive to a field or tradition of research. Such disruptive influences are characterized by citations to a focal paper, but not its cited references. In this study, we are interested in the question of convergent validity. We used external criteria of newness to examine convergent validity: In the postpublication peer review system of F1000Prime, experts assess papers whether the reported research fulfills these criteria (e.g., reports new findings). This study is based on 120,179 papers from F1000Prime published between 2000 and 2016. In the first part of the study we discuss the indicators. Based on the insights from the discussion, we propose alternate variants of disruption indicators. In the second part, we investigate the convergent validity of the indicators and the (possibly) improved variants. Although the results of a factor analysis show that the different variants measure similar dimensions, the results of regression analyses reveal that one variant ( DI 5 ) performs slightly better than the others

Convergent evolution · Convergent validity · Correlation · Academic Publishing and Open Access · Innovation, Sustainability, Human-Machine Systems · scientometrics and bibliometrics research

Tools

Open DOIOpen Access