Improving the Reliability of Physician Performance Assessment

BACKGROUND: The proliferation of efforts to assess physician performance underscore the need to improve the reliability of physician-level quality measures. OBJECTIVE: Using diabetes care as a model, to address 2 key issues in creating reliable physician-level quality performance scores: estimating the physician effect on quality and creating composite measures. DESIGN: Retrospective longitudinal observational study. SUBJECTS: A national sample of physicians (n = 210) their patients with diabetes (n = 7574) participating in the National Committee on Quality Assurance-American Diabetes Association's Diabetes Provider Recognition Program. MEASURES: Using 11 diabetes process and intermediate outcome quality measures abstracted from the medical records of participants, we tested each measure for the magnitude of physician-level variation (the physician effect or "thumbprint"). We then combined measures with a substantial physician effect into a composite, physician-level diabetes quality score and tested its reliability. RESULTS: We identified the lowest target values for each outcome measure for which there was a recognizable "physician thumbprint" (ie, intraclass correlation coefficient > or =0.30) to create a composite performance score. The internal consistency reliability (Cronbach's alpha) of the composite score, created by combining the process and outcome measures with an intraclass correlation coefficient > or =0.30, exceeded 0.80. The standard errors of the composite case-mix adjusted score were sufficiently small to discriminate those physicians scoring in the highest from those scoring in the lowest quartiles of the quality of care distribution with no overlap. CONCLUSIONS: We conclude that the aggregation of well-tested quality measures that maximize the "physician effect" into a composite measure yields reliable physician-level quality of care scores for patients with diabetes

Cronbach's alpha · Family medicine · Intraclass correlation · Medical record · Observational study · Operations management · Psychometrics · Quality (philosophy) · Quality management · Quality Score · Quartile · Reliability (semiconductor) · Clinical Psychology · Healthcare Policy and Management · Internal Medicine · Medicine · Patient Satisfaction in Healthcare · Primary Care and Health Outcomes

• Profiling Individual Physicians Using Administrative Data From a Single Insurer

  Frank Eijkenaar, René C J A Van Vliet•Medical Care•2013

  Incluir item

• Monitoring Modifiable Cardiovascular Risk in Type 2 Diabetes Care in General Practice

  Geert Goderis, Liesbeth Borgermans et al.•Medical Care•2010

  Incluir item

• Reliability of Medical Group and Physician Performance Measurement in the Primary Care Setting

  Thomas D Sequist, Eric C Schneider et al.•Medical Care•2011

  Incluir item

• Impact of Adherence to Quality Measures for Localized Prostate Cancer on Patient-reported Health-related Quality of Life Outcomes, Patient Satisfaction, and Treatment-related Complications

  William Sohn, Matthew J Resnick et al.•Medical Care•2016

  Incluir item

• Improving the Reliability of Physician “Report Cards”

  Kimberly Smith, Kimberly A Smith et al.•Medical Care•2013

  Incluir item

• Precision of Individual and Composite Performance Scores

  Arna L van Doorn-Klomberg, Jozé C C Braspenning et al.•Medical Care•2013

  Incluir item

• The Reliability of Medical Group Performance Measurement in a Single Insurer’s Pay for Performance Program

  Hector P Rodriguez, Lisa Perry et al.•Medical Care•2012

  Incluir item

• Improving the Quality of Quality Measurement

  Monika M Safford•Medical Care•2009

  Incluir item

• Improving the quality of health care for chronic conditions

  J E Epping-Jordan, JoAnne E Epping‐Jordan•BMJ Quality & Safety•2004

  Incluir item

• Forming inferences about some intraclass correlation coefficients.

  Kenneth O Mcgraw, Senya Wong et al.•Psychological Methods•1996

  Incluir item

• Measurement reliability and agreement in psychiatry

  Open Access•Patrick E Shrout•Statistical Methods in Medical…•1998

  Incluir item

• Racial Disparities in the Quality of Care for Enrollees in Medicare Managed Care

  Eric C Schneider•JAMA•2002

  Incluir item

• The predictive value of self assessed general, physical, and mental health on functional decline and mortality in older adults

  Yunhwan Lee•Journal of Epidemiology and…•2000

  Incluir item

• Is More Information Better? The Effects of “Report Cards” on Health Care Providers

  David Dranove, Daniel P Kessler et al.•Journal of Political Economy•2003

  Incluir item

• Racial Disparities in Diabetes Care Processes, Outcomes, and Treatment Intensity

  Michele Heisler, Dylan M Smith et al.•Medical Care•2003

  Incluir item

• Physician Performance Assessment

  Patricia H Parkerton, Dean G Smith et al.•Medical Care•2003

  Incluir item

• Provider Profiling and Quality Improvement Efforts in Coronary Artery Bypass Graft Surgery

  Edward L Hannan, Mary S Vaughn Sarrazin et al.•Medical Care•2003

  Incluir item

• Intraclass correlations

  Patrick E Shrout, Joseph L Flei et al.•Psychological Bulletin•1979

  Incluir item