SDCF

Semi-automatically structured dataset of citation functions

Bibliographic Data

ID21441623
AuthorsSetio Basuki (0000-0001-6631-9880, Toyohashi University of Technology, corresponding author), Masatoshi Tsuchiya (0000-0003-1862-8149, Toyohashi University of Technology)
Year2022
Volume127
Issue8
Pages4569-4608
Publication date2022-08-01
Open AccessYes
TypeARTICLE
VenueScientometrics (JOURNAL)
Journal identifiersISSN: 0138-9130 • E-ISSN: 1588-2861
PublisherSpringer Science and Business Media LLC (PUBLISHER)
DOI10.1007/s11192-022-04471-x
OpenAlexW4286499283
LanguageEN
Citations received1
References cited39

There is increasing research interest in the automatic detection of citation functions, which is why authors of academic papers cite previous works. A machine learning approach for such a task requires a large dataset consisting of varied labels of citation functions. However, existing datasets contain a few instances and a limited number of labels. Furthermore, most labels have been built using narrow research fields. Addressing these issues, this paper proposes a semiautomatic approach to develop a large dataset of citation functions based on two types of datasets. The first type contains 5668 manually labeled instances to develop a new labeling scheme of citation functions, and the second type is the final dataset that is built automatically. Our labeling scheme covers papers from various areas of computer science, resulting in five coarse labels and 21 fine-grained labels. To validate the scheme, two annotators were employed for annotation experiments on 421 instances that produced Cohen’s Kappa values of 0.85 for coarse labels and 0.71 for fine-grained labels. Following this, we performed two classification stages, i.e., filtering, and fine-grained to build models using the first dataset. The classification followed several scenarios, including active learning (AL) in a low-resource setting. Our experiments show that Bidirectional Encoder Representations from Transformers (BERT)-based AL achieved 90.29% accuracy, which outperformed other methods in the filtering stage. In the fine-grained stage, the SciBERT-based AL strategy achieved a competitive 81.15% accuracy, which was slightly lower than the non-AL strategy. These results show that the AL is promising since it requires less than half of the dataset. Considering the number of labels, this paper released the largest dataset consisting of 1,840,815 instances

Annotation · Citation · Data mining · Encoder · Information retrieval · Machine learning · Scheme (mathematics) · Task (project management) · World Wide Web · Artificial Intelligence · Computer Science · Mathematics · Natural Language Processing Techniques · Software Engineering Research · Topic Modeling

Tools

Open DOIOpen Access