Issues in the Use of Kappa to Estimate Reliability

Selected issues regarding the application of the kappa statistic in assessing the reliability of measures to classify patients are discussed. Data are used to illustrate how the methods used to estimate reliability can influence the resulting coefficient. Suggestions for a more comprehensive approach to the study of the psychometric properties of measures of patient classification are then offered

Cohen's kappa · Data mining · Kappa · Machine learning · Reliability (semiconductor) · Reliability engineering · Statistic · Statistics · Computer Science · Engineering · Hemodynamic Monitoring and Therapy · Mathematics · Reliability and Agreement in Measurement · Statistical Methods in Epidemiology

• Severity of Illness Index and the Adverse Patient Occurrence Index

  Dale N Schumacher, Barbara Parker et al.•Medical Care•1987

  Incluir item

• A Proposed Prototype for Identifying and Correcting Sources of Measurement Error in Classification Systems

  Darlene A McKenzie, Darlene McKenzie•Medical Care•1991

  Incluir item

• Characterization of Geriatric Drug-Related Hospital Readmissions

  Lisa A Bero, Lisa Bero et al.•Medical Care•1991

  Incluir item

• Best Practices in Quantitative Methods

  Jason W Osborne, Jason Osborne•Best Practices in Quantitative…•2008

  Incluir item

• Two Quantitative Approaches for Estimating Content Validity

  Open Access•Christine A Wynd, Bruce Schmidt et al.•Western Journal of Nursing Research•2003

  Incluir item

• Interrater agreement and interrater reliability

  Open Access•Natasa Gisev, J Simon Bell et al.•Research in Social and…•2013

  Incluir item

• The Kappa Statistic in Reliability Studies

  Joe Sims, Julius Sim et al.•Physical Therapy•2005

  Incluir item

• Resampling Probability Values for Weighted Kappa with Multiple Raters

  Open Access•Paul W Mielke, Kenneth J Berry et al.•Psychological Reports•2008

  Incluir item

• An Application of Hierarchical Kappa-type Statistics in the Assessment of Majority Agreement among Multiple Observers

  J R Landis, Gary G Koch•Biometrics•1977

  Incluir item

• Interrater reliability and agreement of subjective judgments.

  Howard E A Tinsley, Howard E Tinsley et al.•Journal of Counseling Psychology•1975

  Incluir item

• On the Methods and Theory of Reliability

  John J Bartko, Will Tom Carpenter et al.•The Journal of Nervous and Mental…•1976

  Incluir item

• The Measurement of Observer Agreement for Categorical Data

  J R Landis, Gary G Koch•Biometrics•1977

  Incluir item

• Coefficient Kappa

  Open Access•Robert L Brennan, Dale J Prediger•Educational and Psychological…•1981

  Incluir item

• The Equivalence of Weighted Kappa and the Intraclass Correlation Coefficient as Measures of Reliability

  Open Access•Joseph L Fleiss, Jacob Cohen•Educational and Psychological…•1973

  Incluir item

• A Coefficient of Agreement for Nominal Scales

  Open Access•Jacob Cohen•Educational and Psychological…•1960

  Incluir item

• Paying hospitals

  C Muller•American Journal of Public Health•1983

  Incluir item

• Measuring Severity of Illness

  Susan D Horn•American Journal of Public Health•1983

  Incluir item

• The Abbreviated Injury Scale and Injury Severity Score

  Ellen J Mackenzie, Sam Shapiro et al.•Medical Care•1985

  Incluir item

• Measuring Severity of Illness

  Susan D Horn, Phoebe D Sharkey et al.•Medical Care•1983

  Incluir item

• The Measurement of Hospital Case Mix

  WANDA W YOUNG, Robert B Swinkola et al.•Medical Care•1982

  Incluir item

• Determining the Generalizability of Rating Scales in Clinical Settings

  William J Evans, C Gene Cayten et al.•Medical Care•1981

  Incluir item

• Measuring Severity of Illness

  Susan D Horn, Bette Chachich et al.•Medical Care•1983

  Incluir item

• The Dependability of Medical Encounter Diagnostic Information

  Dennis A Revicki•Medical Care•1984

  Incluir item

• Integration and generalization of kappas for multiple raters

  Anthony J Conger•Psychological Bulletin•1980

  Incluir item

• Measuring nominal scale agreement among many raters

  Joseph L Flei, Joseph L Fleiss•Psychological Bulletin•1971

  Incluir item