Psychometric validation of the reconstructed version of the assessment of reasoning tool

Bibliographic Data

ID21447833
AuthorsSatid Thammasitboon (0000-0001-6896-092X, Baylor College of Medicine, corresponding author), Moushumi Sur (Baylor College of Medicine), Joseph J Rencic (Department of Medicine, Boston University School of Medicine, Boston, MA, USA), Joseph Rencic (0000-0002-2598-3299, Boston University), Gurpreet Dhaliwal (0000-0002-6134-5913, University of California, San Francisco), Shelley Kumar (0000-0002-0927-6762, Texas Children's Hospital), Suresh Sundaram (0000-0001-8502-6848, Department of Administration, Alfred Lerner College of Business & Economics, University of Delaware, Newark, DE, USA), Parthasarathy Krishnamurthy (0000-0001-8924-0542, Baylor College of Medicine)
Year2021
Volume43
Issue2
Pages168-173
Publication date2021-02-01
Open AccessNo
TypeARTICLE
VenueMedical Teacher (JOURNAL)
Journal identifiersISSN: 0142-159X • E-ISSN: 1466-187X
PublisherInforma UK Limited (PUBLISHER • GB)
DOI10.1080/0142159x.2020.1830960
PMID33073665
OpenAlexW3092936890
LanguageEN
Citations received1
References cited19

BACKGROUND: Assessing learners' competence in diagnostic reasoning is challenging and unstandardized in medical education. We developed a theory-informed, behaviorally anchored rubric, the Assessment of Reasoning Tool (ART), with content and response process validity. This study gathered evidence to support the internal structure and the interpretation of measurements derived from this tool. METHODS: We derived a reconstructed version of ART (ART-R) as a 15-item, 5-point Likert scale using the ART domains and descriptors. A psychometric evaluation was performed. We created 18 video variations of learner oral presentations, portraying different performance levels of the ART-R. RESULTS: 152 faculty viewed two videos and rated the learner globally and then using the ART-R. The confirmatory factor analysis showed a favorable comparative fit index = 0.99, root mean square error of approximation = 0.097, and standardized root mean square residual = 0.026. The five domains, hypothesis-directed information gathering, problem representation, prioritized differential diagnosis, diagnostic evaluation, and awareness of cognitive tendencies/emotional factors, had high internal consistency. The total score for each domain had a positive association with the global assessment of diagnostic reasoning. CONCLUSIONS: Our findings provide validity evidence for the ART-R as an assessment tool with five theoretical domains, internal consistency, and association with global assessment

Cognition · Cognitive psychology · Competence (human resources) · Confirmatory factor analysis · Content validity · Convergent validity · Developmental psychology · Internal consistency · Likert scale · Machine learning · Mathematics education · Psychometrics · Rubric · Structural equation modeling · Clinical Psychology · Clinical Reasoning and Diagnostic Skills · Computer Science · Innovations in Medical Education · Psychology · Simulation-Based Education in Healthcare · Social Psychology

Tools

Open DOI