Development and preliminary inter-rater reliability of the new PROOF tool to measure fidelity of problem-solving therapy for depression delivered by non-specialists in a low-resource African setting
Bibliographic Data
| ID | 7148877 |
|---|---|
| Authors | Lily Cooke (0009-0003-9529-0249, King's College London, corresponding author), Tarisai Bere (University of Zimbabwe), Amelia M Stanton (0000-0001-5918-6645), Amelia Stanton, Walter Mangezi (0000-0002-8889-0487, University of Zimbabwe), Steven A Safren (0000-0002-0121-0806, University of Miami), Tsitsi Mawere (University of Zimbabwe), Lene Søndergaard Andersen (0000-0002-9643-4502, University of Copenhagen), Christina Psaros (0000-0002-0705-2239, Massachusetts General Hospital), Samantha M Mcketchnie (0000-0001-8004-4430, Harvard University Press), Meghana Vagwala (0000-0001-8244-768X, Harvard University Press), Kia-Chong Chua, Kia‐Chong Chua (0000-0002-6693-6333, King's College London), Conall O’cleirigh (0000-0002-5639-5043, Harvard University Press), Aya Mitani (0000-0002-0373-5032, University of Toronto), Melanie Abas (0000-0003-1716-5584, King's College London) |
| Year | 2025 |
| Volume | 12 |
| Pages | e98-e98 |
| Publication date | 2025-01-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Cambridge Prisms Global Mental Health (JOURNAL) |
| Journal identifiers | ISSN: 2054-4251 • E-ISSN: 2054-4251 |
| Publisher | Cambridge University Press (CUP) (PUBLISHER) |
| DOI | 10.1017/gmh.2025.10034 |
| PMID | 40927224 |
| OpenAlex | W4414049566 |
| Language | EN |
| References cited | 17 |
Problem-solving therapy (PST) is a brief psychological intervention often implemented for depression. Currently, there are no tools with well-evidenced reliability to measure PST fidelity. This pilot study aimed to measure the inter-rater reliability and agreement of the Pro blem-S o lving Therapy F idelity (PROOF) scale, comprising binary 14-item adherence and an 8-item competence subscales. Transcripts were from the TENDAI trial, a Zimbabwe-based PST intervention for depression and medication adherence. Seven transcripts were each rated by seven specialists, and two transcripts were each rated by two non-specialists. Inter-rater agreement was assessed using percent agreement and inter-rater reliability was assessed using Gwet’s AC 1 . The PROOF subscales demonstrated promising inter-rater agreement among specialists (adherence = 90.4%, competence = 82.5%) and non-specialists (adherence = 92.9%, competence = 68.8%). Inter-rater reliability analyses yielded a Gwet’s AC 1 of 0.411–0.778 and 0.619–0.959 for adherence and competence among specialists, and 0.529–1.00 for adherence in non-specialists. The PROOF scale has the potential to fill the gap of fidelity tools for PST delivery
Fidelity · Proof of concept · Psychometrics · Problem Solving Skills Development
A comparison of Cohen’s Kappa and Gwet’s AC1 when calculating inter-rater reliability coefficients
Computing Inter-Rater Reliability for Observational Data
Research electronic data capture (Redcap)—A metadata-driven methodology and workflow process for providing translational research informatics support
The Redcap consortium
Interrater reliability
Development of a scoring system for non-specialist ratings of clinical competence in global mental health
The Kappa Statistic
Lay Health Workers’ Experience of Delivering a Problem Solving Therapy Intervention for Common Mental Disorders Among People Living with HIV
| Citation velocity | historical |
|---|---|
| Highly cited | No |