Two Quantitative Approaches for Estimating Content Validity

Bibliographic Data

ID23366972
AuthorsChristine A Wynd (University of Akron College of Nursing), Bruce Schmidt (Cleveland Clinic Akron General), Michelle Atkins Schaefer (914th Combat Support Hospital, Upper Arlington, Ohio)
Year2003
Volume25
Issue5
Pages508-518
Publication date2003-08-01
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueWestern Journal of Nursing Research (JOURNAL)
Journal identifiersISSN: 0193-9459 • E-ISSN: 1552-8456
PublisherSAGE Publications (PUBLISHER • US)
DOI10.1177/0193945903252998
PMID12955968
OpenAlexW2129698558
LanguageEN
Citations received61
References cited26

Instrument content validity is often established through qualitative expert reviews, yet quantitative analysis of reviewer agreements is also advocated in the literature.Two quantitative approaches to content validity estimations were compared and contrasted using a newly developed instrument called the Osteoporosis Risk Assessment Tool (ORAT).Data obtained from a panel of eight expert judges were analyzed. A Content Validity Index (CVI) initially determined that only one item lacked interrater proportion agreement about its relevance to the instrument as a whole (CVI = 0.57). Concern that higher proportion agreement ratings might be due to random chance stimulated further analysis using a multirater kappa coefficient of agreement. An additional seven items had low kappas, ranging from 0.29 to 0.48 and indicating poor agreement among the experts. The findings supported the elimination or revision of eight items. Pros and cons to using both proportion agreement and kappa coefficient analysis are examined.

Cohen's kappa · Content (measure theory) · Content analysis · Content validity · Developmental psychology · Inter-rater reliability · Kappa · Psychometrics · Rating scale · Social science · Statistics · Clinical Psychology · Delphi Technique in Research · Health Education and Validation · Mathematics · Psychology · Reliability and Agreement in Measurement