Identifying Poor-Quality Hospitals

OBJECTIVES: Many groups involved in health care are very interested in using external quality indices, such as risk-adjusted mortality rates, to examine hospital quality. The authors evaluated the feasibility of using mortality rates for medical diagnoses to identify poor-quality hospitals. METHODS: The Monte Carlo simulation model was used to examine whether mortality rates could distinguish 172 average-quality hospitals from 19 poor-quality hospitals (5% versus 25% of deaths being preventable, respectively), using the largest diagnosis-related groups (DRGs) for cardiac, gastrointestinal, cerebrovascular, and pulmonary diseases as well as an aggregate of all medical DRGs. Discharge counts and observed death rates for all 191 Michigan hospitals were obtained from the Michigan Inpatient Database. Positive predictive value (PPV), sensitivity, and area under the receiver operating characteristic curve were calculated for mortality outlier status as an indicator of poor-quality hospitals. Sensitivity analysis was performed under varying assumptions about the time period of evaluation, quality differences between hospitals, and unmeasured variability in hospital casemix. RESULTS: For individual DRG groups, mortality rates were a poor measure of quality, even using the optimistic assumption of perfect casemix adjustment. For acute myocardial infarction, high mortality rate outlier status (using 2 years of data and a 0.05 probability cutoff) had a PPV of only 24%, thus, more than three fourths of those labeled poor-quality hospitals (high mortality rate outliers) actually would have average quality. If we aggregate all medical DRGs and continue to assume very large quality differences and perfect casemix adjustment, the sensitivity for detecting poor-quality hospitals is 35% and PPV is 52%. Even for this extreme case, the PPV is very sensitive to introduction of small amounts of unmeasured casemix differences between hospitals. CONCLUSION: Although they may be useful for some surgical diagnoses, DRG-specific hospital mortality rates probably cannot accurately detect poor-quality outliers for medical diagnoses. Even collapsing to all medical DRGs, hospital mortality rates seem unlikely to be accurate predictors of poor quality, and punitive measures based on high mortality rates frequently would penalize good or average hospitals

Cutoff · Health care · Intensive care medicine · Medical diagnosis · Medical emergency · Mortality rate · Myocardial infarction · Outlier · Quality (philosophy) · Statistics · Emergency Medicine · Internal Medicine · Medical Coding and Health Information · Medicine · Patient Satisfaction in Healthcare · Primary Care and Health Outcomes · Surgery

• Effect of Choice of Estimation Method on Inter-Hospital Mortality Rate Comparisons

  Patricia Kipnis, Gabriel J Escobar et al.•Medical Care•2010

  Añadir item

• Measuring Quality

  Arlene S Ash, Arlene Ash•Medical Care•2008

  Añadir item

• Accuracy of Risk-Adjusted Mortality Rate As a Measure of Hospital Quality of Care

  J W Thomas, Timothy P Hofer•Medical Care•1999

  Añadir item

• Consumer Response to Quality Information

  Michael E Chernew, Michael Chernew et al.•Medical Care•1998

  Añadir item

• Access to Clinically-Detailed Patient Information

  Rodney A Hayward•Medical Care•2008

  Añadir item

• Reducing Random Variation in Reported Rates of Pressure Ulcer Development

  Dan R Berlowitz, Jennifer J Anderson et al.•Medical Care•1998

  Añadir item

• Identifying Poor-Quality Hospitals with Mortality Rates

  Arlene S Ash, Arlene Ash•Medical Care•1996

  Añadir item

• A New Casemix Adjustment Index for Hospital Mortality Among Patients With Congestive Heart Failure

  Carisi A Polanczyk, Luís Eduardo Paim Rohde et al.•Medical Care•1998

  Añadir item

• Risk-Adjusting Hospital Inpatient Mortality Using Automated Inpatient, Outpatient, and Laboratory Databases

  Gabriel J Escobar, J Greene et al.•Medical Care•2008

  Añadir item

• Are Mortality Rates for Different Operations Related

  Justin B Dimick, Douglas O Staiger et al.•Medical Care•2006

  Añadir item

• A morte de idosos na Clínica Santa Genoveva, Rio de Janeiro

  Open Access•Henrique L Guerra, Sandhi Maria Barreto et al.•Cadernos de Saude Publica•2000

  Añadir item

• Validity of the risk adjustment approach to compare outcomes

  Open Access•Letícia Krauss Silva•Cadernos de Saude Publica•2003

  Añadir item

• Hospital Characteristics and Mortality Rates

  Arthur J Hartz, Henry Krakauer et al.•New England Journal of Medicine•1989

  Añadir item

• The meaning and use of the area under a receiver operating characteristic (ROC) curve.

  J A Hanley, Barbara J Mcneil•Radiology•1982

  Añadir item

• Efficacy of Statistical Outlier Analysis for Monitoring Quality of Care

  Kurt D Gillis, Jesse S Hixson•Journal of Business and Economic…•1991

  Añadir item

• Assessing providers of coronary revascularization

  Arthur J Hartz, Evelyn M Kuhn et al.•American Journal of Public Health•1992

  Añadir item

• Adjusted hospital death rates

  Robert W Dubois, Robert H Brook et al.•American Journal of Public Health•1987

  Añadir item

• An Evaluation of Generic Screens for Poor Quality of Hospital Care on a General Medicine Service

  Rodney A Hayward, Annette M Bernard et al.•Medical Care•1993

  Añadir item

• Socioeconomic Influence on Small Area Hospital Utilization

  Laurence F McMahon, Robert A Wolfe et al.•Medical Care•1993

  Añadir item

• Predicting In-Hospital Mortality

  Lisa I Iezzoni, Arlene S Ash et al.•Medical Care•1992

  Añadir item

• Clinical Versus Administrative Data Bases for Cabg Surgery

  Edward L Hannan, Harold Kilburn et al.•Medical Care•1992

  Añadir item

• Reliability and Validity of Judgments Concerning Adverse Events Suffered by Hospitalized Patients

  Troyen A Brennan, Russsell J Localio et al.•Medical Care•1989

  Añadir item