Artificial Intelligence (AI)-Assisted Coding for Competency-Based Skill Improvement
A Case Study of Motivational Interviewing Skill Development in Behavioral Health Providers-in-Training
Bibliographic Data
| ID | 21619929 |
|---|---|
| Authors | Cortny Stark (0000-0003-4685-5808, PhD, assistant professor, The University of New Mexico, Albuquerque, USA, corresponding author), Kade Stanzilis (MA, graduate researcher, University of Colorado Colorado Springs, USA), Simone Plauche (Master’s in counseling student, graduate researcher, University of Colorado Colorado Springs, USA), Amador Palacios (0000-0001-7684-0880, PhD, assistant professor, University of Colorado Colorado Springs, USA), Alfredo F Palacios (University of Colorado Colorado Springs) |
| Year | 2025 |
| Publication date | 2025-12-31 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Families in Society The Journal of Contemporary Social Services (JOURNAL) |
| Journal identifiers | ISSN: 1044-3894 • E-ISSN: 1945-1350 |
| Publisher | SAGE Publications (PUBLISHER • US) |
| DOI | 10.1177/10443894251378612 |
| OpenAlex | W7117699694 |
| Language | EN |
| References cited | 37 |
Coding of session transcripts is a key strategy in assessing counselors’ and social workers’ fidelity to the motivational interviewing (MI) approach. Artificial intelligence (AI) shows promise in identifying patterns in session transcripts associated with MI proficiency. This study utilizes a multiple instrumental case study approach to compare and contrast intercoder reliability for Atlas.ti, versus human researchers, for six session transcripts using the Motivational Interviewing Competency Assessment (MICA). Atlas.ti’s scores and qualitative feedback were compared to human researchers, and interrater reliability calculated using the Intraclass Correlation Coefficient. Results indicate significant agreement between researchers and AI scoring of intention and strategy, as well as the composite score. The difference in reflection-to-question ratio scoring highlights a difference in AI’s ability to attend to the nuanced nature of utilizing MI skills and strategies to enhance client/patient motivation
Fidelity · Inter-rater reliability · Intraclass correlation · Motivational interviewing · Artificial Intelligence in Healthcare and Education · Counseling Practices and Supervision · Digital Mental Health Interventions
The Motivational Interviewing Treatment Integrity Code (Miti 4)
A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research
Implementing motivational interviewing in adult mental health social work practice
Grade inflation versus grade improvement
Waiting for a digital therapist
Impact of Motivational Interviewing by Social Workers on Service Users
A Meta-Analysis of Motivational Interviewing
Training Incoming Social Work Students in Motivational Interviewing Skills
| Citation velocity | historical |
|---|---|
| Highly cited | No |