Profiling Individual Physicians Using Administrative Data From a Single Insurer

BACKGROUND: Individual physicians are increasingly being subjected to comparative performance assessments. When single-insurer data are used to profile individual physicians' performance, reliable measurements are uncertain because of small sample sizes. METHODS: Administrative data (2006-2008) from a Dutch insurer are used to examine variation in general practitioners' (GPs) performance on expenses (5 measures), utilization of hospital care (2 measures), and clinical quality for diabetes and chronic obstructive pulmonary disease (6 measures). Unadjusted and adjusted multilevel models are used to separate total variance in between-GP and within-GP components. The components are used to calculate intraclass correlation coefficients (ICCs), reliability, and sample size requirements at common reliability thresholds. RESULTS: Average ICCs varied between 0.07% (hospital admissions) and 8.34% (physiotherapy for chronic obstructive pulmonary disease patients). Risk-adjustment often greatly changed the relative size of variance components and often led to lower ICCs. In addition, ICCs and thus reliability generally decreased over time. Eight measures had reliabilities > 0.70, and 3 of these (all GP-related expenses) > 0.90. Measures related to utilization of hospital care had reliabilities < 0.60 or even 0.50. For 5 measures, the vast majority of GPs had sufficient patients to reach 0.70 reliability. At a reliability of 0.90, however, there were no measures for which all GPs met the sample size requirements. CONCLUSIONS: Reliable measurement of individual physicians' performance using single-purchaser data is challenging. For most measures reliability was insufficient to allow for high-stakes applications or even any application of profiling. Future research should continue to explore methods for enhancing the reliability of individual physicians' profiles

Intraclass correlation · Psychometrics · Reliability (semiconductor) · Sample (material) · Sample size determination · Statistics · Variance (accounting) · Healthcare Policy and Management · Mathematics · Medicine · Patient Satisfaction in Healthcare · Primary Care and Health Outcomes

• Prior Use of Durable Medical Equipment as a Risk Adjuster for Health-Based Capitation

  Open Access•Richard C van Kleef, René C J A Van Vliet•INQUIRY The Journal of Health…•2010

  Incluir item

• Health-Based Risk Adjustment

  Open Access•Femmeke J Prinsze, René C J A Van Vliet•INQUIRY The Journal of Health…•2007

  Incluir item

• Reliability of Medical Group and Physician Performance Measurement in the Primary Care Setting

  Thomas D Sequist, Eric C Schneider et al.•Medical Care•2011

  Incluir item

• Reducing Random Variation in Reported Rates of Pressure Ulcer Development

  Dan R Berlowitz, Jennifer J Anderson et al.•Medical Care•1998

  Incluir item

• Are There Detectable Differences in Quality of Care or Outcome of Depression Across Primary Care Providers

  Wayne Katon, Carolyn M Rutter et al.•Medical Care•2000

  Incluir item

• Improving the Reliability of Physician Performance Assessment

  Sherrie H Kaplan, James L Griffith et al.•Medical Care•2009

  Incluir item

• Meaningful Variation in Performance

  Joseph V Selby, Julie A Schmittdiel et al.•Medical Care•2010

  Incluir item

• How can Health Care Organizations be Reliably Compared

  Georgios Lyratzopoulos, Marc N Elliott et al.•Medical Care•2011

  Incluir item

• Meaningful Variation in Performance

  Vicki Fung, Julie A Schmittdiel et al.•Medical Care•2010

  Incluir item

• Risk-adjusted Payment and Performance Assessment for Primary Care

  Arlene S Ash, Randall P Ellis•Medical Care•2012

  Incluir item