Validation of the ICO-Ocex tool for competency-based assessment in ophthalmology residency programs
Bibliographic Data
| ID | 22424802 |
|---|---|
| Authors | Ambreen Gull (0000-0003-2992-2116, Rawalpindi Medical University, corresponding author), Rahila Yasmin (Riphah International University), Usman Mahboob (0000-0002-4756-6787, Khyber Medical University), Karl C Golnik (0000-0002-8822-5989, Barrow Neurological Institute), Muhammad Moin (0009-0007-4893-2754, King Edward Medical University) |
| Year | 2026 |
| Volume | 26 |
| Issue | 1 |
| Publication date | 2026-04-09 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | BMC Medical Education (JOURNAL) |
| Journal identifiers | ISSN: 1472-6920 • E-ISSN: 1472-6920 |
| Publisher | Springer Science and Business Media LLC (PUBLISHER) |
| DOI | 10.1186/s12909-026-09154-2 |
| PMID | 41957784 |
| OpenAlex | W7152504462 |
| Language | EN |
| References cited | 32 |
BACKGROUND: Competence in medical training requires trainees to independently demonstrate practical skills. The ICO-OCEX (International Council of Ophthalmology-Ophthalmic Clinical Evaluation Exercise) tool, an improved version of OCEX (Ophthalmic clinical evaluation exercise), employs behaviorally descriptive milestones based on the Dreyfus model to address inconsistent grading interpretations and effectively assess ophthalmology residents’ competencies through workplace-based assessments. PURPOSE: To determine the validity and reliability of ICO-OCEX formative assessment tool for ophthalmology postgraduate residents. METHODS: A multiphase mixed-method instrument developmental approach was used. Quantitative content validation was conducted using the modified Delphi technique, while cognitive pretesting assessed response process validity. Pilot testing was carried out to evaluate construct validity, as well as reliability and inter-rater reliability. RESULTS: The modified Delphi technique achieved consensus on all 33 items, confirming stability and a scale content validity index of 0.97. Cognitive interviews led to revisions, resulting in a 34-item instrument. Confirmatory factor analysis indicated a good to excellent model fit. Cronbach’s alpha was 0.995 and Fleiss Kappa showed moderate to substantial inter-rater reliability. CONCLUSION: The ICO-OCEX tool, consisting of 34 items measuring four constructs on a three-point Dreyfus scale, demonstrated acceptable validity and reliability. It may be useful for formative assessment and supporting competency development in ophthalmology residency programs, particularly when faculty receive training for consistent application.
Cognition · Confirmatory factor analysis · Construct validity · Content validity · Delphi method · Educational measurement · Formative assessment · Intraocular Surgery and Lenses · Ophthalmology and Visual Health Research · Ophthalmology and Visual Impairment Studies
Developing questionnaires for educational research
Factor analysis and scale revision.
Factor Analysis as a Tool for Survey Analysis
Cutoff criteria for fit indexes in covariance structure analysis
Interrater reliability
Exploring the use of rating scales with entrustment anchors in workplace‐based assessment
Know thyself
Stability and agreement criteria for the termination of Delphi studies
How Cognitive Interviewing can Provide Validity Evidence of the Response Processes to Scale Items
Validity and the Consequences of Test Interpretation and Use
| Citation velocity | historical |
|---|---|
| Highly cited | No |