Development, field testing, and initial validation of an adherence rubric for caregiver coaching

Bibliographic Data

ID22167096
AuthorsScott D Tomchek, Scott Tomchek (0000-0003-4642-0177, University of Louisville, corresponding author), Serena Wheeler (0000-0001-7643-0849, University of Louisville), Cybil Cheek (University of Louisville), Lauren Little (0000-0003-0995-0531, Rush University), Winnie Dunn (0000-0001-8697-0590, University of Missouri Health System)
Year2024
Volume9
Publication date2024-10-22
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueFrontiers in Education (JOURNAL)
Journal identifiersISSN: 2504-284X • E-ISSN: 2504-284X
PublisherFrontiers Media SA (PUBLISHER • CH)
DOI10.3389/feduc.2024.1324118
OpenAlexW4403659880
LanguageEN
Citations received1
References cited38

Introduction The translation and use of evidence-based practices (EBPs) within early intervention (EI) systems presents challenges. The Office of Special Education (OSEP) has emphasized results-driven accountability to expand state accountability from compliance to also include quality services that align with EBPs. OSEP’s results-driven accountability provided states the opportunity to design State Systemic Improvement Plans (SSIP) to strengthen the quality of EI services by increasing the capacity of EI systems to implement, scale-up, and sustain use of EBPs. Caregiver coaching is widely accepted as an EBP within EI settings, yet uptake and fidelity to coaching practices remains limited. Such widespread implementation of caregiver coaching is partially limited by a lack of measurement tools that operationalize behaviors consistent with coaching. In this study, we describe the development of the Kentucky Coaching Adherence Rubric-Revised (KCAR-R) and psychometric testing of the instrument. Methods We developed and tested the KCAR-R to measure fidelity of coaching practices within a state-wide professional development program, the Coaching in Early Intervention Training and Mentorship Program. We define operational elements of the KCAR-R and rubric design elements related to: creators; users and uses; specificity, judgment complexity; evaluative criteria; quality levels; quality definitions; scoring strategies; presentation; explanation; quality processes; accompanying feedback information; secrecy; and exemplars. With regard to psychometric validation of the KCAR-R, interrater reliability was analyzed using intraclass correlation coefficients across eight raters and 301 randomly selected video submissions. Internal consistency was evaluated using Cronbach’s alpha across 429 video submissions. Results Results showed 0.987 agreement, indicating excellent interrater reliability; item level internal consistency values ranged from 0.860 to 0.882. for scale if item deleted and 0.834 for the total scale. Discussion Findings from this study showed that the KCAR-R operationalized behaviors that exemplify caregiver coaching and may serve as a resource for other states or programs to document the quality and fidelity of evidence-based EI services. To influence EI provider practices at a systems level, we used implementation science to guide our work and provide examples of how EI systems seeking ways to create sustainable quality services may build upon our approach

Coaching · Medical physics · Pedagogy · Psychotherapist · Rubric · Software engineering · Systems engineering · Coaching Methods and Impact · Computer Science · Educational and Psychological Assessments · Engineering · Family and Disability Support Research · Mathematics · Medicine · Psychology

Tools

Open DOIOpen Access