How Reliable and Valid are the Evaluations of Digital Competence in Higher Education

A Systematic Mapping Study

Bibliographic Data

ID16910348
AuthorsRafael Saltos-Rivas (0000-0003-0458-7518, Universidad Técnica de Manabí), Pavel Novoa‐Hernández (0000-0003-3267-6753, Universidad Católica del Norte, corresponding author), Rodolfo Siller Rodríguez (0000-0002-9447-9336, University of Córdoba), Rocío Serrano Rodríguez
Year2022
Volume12
Issue1
Publication date2022-01-01
Open AccessYes
TypeARTICLE
VenueSAGE Open (JOURNAL)
Journal identifiersISSN: 2158-2440 • E-ISSN: 2158-2440
PublisherSAGE Publications (PUBLISHER • US)
DOI10.1177/21582440211068492
OpenAlexW4206509286
LanguageEN
Citations received2
References cited35

Evaluating digital competencies has become a topic of growing interest in recent years. Although several reviews and studies have summarized the main elements of progress and shortcomings in this area, some issues are yet to be explored. Very little information is available about the ways of ensuring the validity and reliability of the instrument used. This study addresses this issue through systematic mapping covering the period from January 2015 to July 2020. Based on 88 primary studies, we conclude that a growing number of studies have emerged over the years; most of them are based on European university students in social science programs; the quality of the journals where they were published is low; and the instruments used are mostly questionnaires and ad-hoc surveys that measure the various dimensions of digital competence based on participants’ perceptions. Out of the eight possible types of quality assessment, studies commonly report only four (one in the case of reliability and three in the case of validity). More than 50% of the studies do not provide evidence of having assessed the reliability and validity at the same time. Although participant discipline resulted in a significantly association with the practice of reporting reliability and validity assessments, no specific dimension explained such association. A non-parametric multivariate analysis reveals, among other interesting patterns, that the practice of not reporting quality assessments is more present in studies where participants belong to the fields of Engineering and Technology, Health Sciences, and Humanities

Data science · Medical education · MEDLINE · Political science · Psychometrics · Systematic review · Validity · Applied Psychology · Clinical Psychology · Computer Science · Digital literacy in education · Educational Innovations and Technology · Medicine · Psychology · Semiconductor materials and interfaces · Social Psychology

Tools

Open DOIOpen Access