A Proposed Prototype for Identifying and Correcting Sources of Measurement Error in Classification Systems

Bibliographic Data

ID9103936
AuthorsDarlene A McKenzie, Darlene McKenzie (Oregon Health & Science University, corresponding author)
Year1991
Volume29
Issue6
Pages521-530
Publication date1991-06-01
Open AccessNo
TypeARTICLE
VenueMedical Care (JOURNAL)
Journal identifiersISSN: 0025-7079 • E-ISSN: 1537-1948
PublisherOvid Technologies (Wolters Kluwer Health) (PUBLISHER)
DOI10.1097/00005650-199106000-00011
PMID1904516
OpenAlexW1964930422
LanguageEN
References cited8

Because many raters are generally involved in the implementation of a patient classification system, interrater reliability is always a concern in the development and use of such a system. In this article, a case example is used to demonstrate a prototype for identifying measurement error introduced at each step in the classification process (assessment, creating summary item responses, and use of these responses for categorization) and to illustrate how this identification may lead to error reduction strategies. The methods of analyses included percent agreement, Kappa, and visual inspection of contingency tables displaying interrater responses to assessment items, summary items, and the placement category. The extent to which raters followed instructions was analyzed by comparing their responses with computer-generated responses across the classification steps. In addition, raters were interviewed regarding their use of the system

Categorization · Cohen's kappa · Contingency table · Data mining · Identification (biology) · Inter-rater reliability · Kappa · Machine learning · Natural language processing · Process (computing) · Rating scale · Reliability (semiconductor) · Statistics · Artificial Intelligence · Computer Science · Mathematics · Psychology · Reliability and Agreement in Measurement

Tools

Open DOISci-Hub