Measuring fidelity of brief cognitive behavior therapy for insomnia

PURPOSE: Measuring fidelity is critical in program evaluations to assess how implementation influences outcomes. Implementer fidelity is comprised of adherence to the treatment content and competence of treatment delivery. Cognitive behavioral therapy for insomnia (CBT-I) is well-established and empirically supported with variants such as brief CBTi (bCBTi) showing promise in reducing both insomnia and depression. The impact of therapist fidelity on treatment outcomes is unknown in part because reliable measures have not been available. We developed measures of therapist fidelity for bCBT-i and assessed the impact of fidelity in the context of a pilot study with veterans in primary care. METHODS/RESULTS: Audio recordings from 23 participants (78 % male) were coded. Therapist adherence measures were created for each session along with a single measure of therapist competence. Inter-rater reliability was established and predictive validity was determined. For total adherence, inter-rater reliability was excellent across sessions (ICC =.73 -.80). The competence measure showed good reliability across all sessions (ICC =.57) and was internally consistent (Cronbach's alpha =.75). There was only 10 % of shared variance between adherence and competence. All of the fidelity measures demonstrated associations with outcomes in the predicted direction; therapist adherence was significantly associated with decreased depression

Cognition · Fidelity · Insomnia · Psychiatry · Reliability (semiconductor) · Reliability engineering · Applied Psychology · Clinical Psychology · Computer Science · Engineering · Psychiatric care and mental health services · Psychology · Schizophrenia research and treatment · Sleep and related disorders

• Management of Chronic Insomnia Disorder in Adults

  Open Access•Amir Qaseem, Devan Kansagara et al.•Annals of Internal Medicine•2016

  Incluir item

• Validity of the Brief Patient Health Questionnaire Mood Scale (PHQ-9) in the general population

  Open Access•Alexandra Martins, Alexandra Martin et al.•General Hospital Psychiatry•2006

  Incluir item

• European guideline for the diagnosis and treatment of insomnia

  Open Access•Dieter Riemann, Chiara Baglioni et al.•Journal of Sleep Research•2017

  Incluir item

• Computing inter‐rater reliability and its variance in the presence of high agreement

  Open Access•Kilem L Gwet•British Journal of Mathematical…•2008

  Incluir item

• Being Inconsistent About Consistency

  David L Streiner•Journal of Personality Assessment•2003

  Incluir item

• Development of a framework and coding system for modifications and adaptations of evidence-based interventions

  Open Access•Shannon Wiltsey Stirman, Christopher J Miller et al.•Implementation Science•2013

  Incluir item

• Validation of the Insomnia Severity Index as an outcome measure for insomnia research

  Open Access•Célyne Bastien•Sleep Medicine•2001

  Incluir item

• High agreement but low Kappa

  Open Access•Alvan R Feinstein, Domenic V Cicchetti•Journal of Clinical Epidemiology•1990

  Incluir item

• Implementation fidelity in community‐based interventions

  Open Access•Susan M Breitenstein, Deborah Gro et al.•Research in Nursing & Health•2010

  Incluir item

• Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology.

  Domenic V Cicchetti•Psychological Assessment•1994

  Incluir item

• The PHQ-9

  Kurt Kroenke, Robert L Spitzer et al.•Journal of General Internal…•2001

  Incluir item

• A Coefficient of Agreement for Nominal Scales

  Open Access•Jacob Cohen•Educational and Psychological…•1960

  Incluir item

• Examining implementer fidelity

  Open Access•Wendi Cross, Jennifer West•Journal of Children s Services•2011

  Incluir item

• The Kappa Statistic

  Open Access•Barbara Di Eugenio, Michael R Gla et al.•Computational Linguistics•2004

  Incluir item