Validation of Diagnostic Groups Based on Health Care Utilization Data Should Adjust for Sampling Strategy

OBJECTIVE: Valid measurement of outcomes such as disease prevalence using health care utilization data is fundamental to the implementation of a "learning health system." Definitions of such outcomes can be complex, based on multiple diagnostic codes. The literature on validating such data demonstrates a lack of awareness of the need for a stratified sampling design and corresponding statistical methods. We propose a method for validating the measurement of diagnostic groups that have: (1) different prevalences of diagnostic codes within the group; and (2) low prevalence. METHODS: We describe an estimation method whereby: (1) low-prevalence diagnostic codes are oversampled, and the positive predictive value (PPV) of the diagnostic group is estimated as a weighted average of the PPV of each diagnostic code; and (2) claims that fall within a low-prevalence diagnostic group are oversampled relative to claims that are not, and bias-adjusted estimators of sensitivity and specificity are generated. APPLICATION: We illustrate our proposed method using an example from population health surveillance in which diagnostic groups are applied to physician claims to identify cases of acute respiratory illness. CONCLUSIONS: Failure to account for the prevalence of each diagnostic code within a diagnostic group leads to the underestimation of the PPV, because low-prevalence diagnostic codes are more likely to be false positives. Failure to adjust for oversampling of claims that fall within the low-prevalence diagnostic group relative to those that do not leads to the overestimation of sensitivity and underestimation of specificity

Diagnosis code · Diagnostic accuracy · Environmental health · False positive paradox · Health care · Population · Sampling (signal processing) · Statistics · Computer Science · Internal Medicine · Mathematics · Medicine · Reliability and Agreement in Measurement · Sepsis Diagnosis and Treatment · Statistical Methods and Bayesian Inference

• Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases

  Open Access•Richard A Deyo, R DEYO•Journal of Clinical Epidemiology•1992

  Incluir item

• The “Meaningful Use” Regulation for Electronic Health Records

  David Blumenthal, Marilyn Tavenner•New England Journal of Medicine•2010

  Incluir item

• Accuracy of syndrome definitions based on diagnoses in physician claims

  Open Access•Geneviève Cadieux, David L Buckeridge et al.•BMC Public Health•2011

  Incluir item

• Accuracy of ICD-9-CM Codes for Identifying Cardiovascular and Stroke Risk Factors

  Elena Birman-Deych, Amy D Waterman et al.•Medical Care•2005

  Incluir item

• Validity of Information on Comorbidity Derived From ICD-9-CCM Administrative Data

  Hude Quan, Gerry A Parsons et al.•Medical Care•2002

  Incluir item

• Using Medicare Claims for Outcomes Research

  Janet B Mitchell, Thomas A Bubolz et al.•Medical Care•1994

  Incluir item

• Telehealth Ontario Detection of Gastrointestinal Illness Outbreaks

  Open Access•Jaelyn Caudle, Jaelyn M Caudle et al.•Canadian Journal of Public Health•2009

  Incluir item