Construct comparability and the limits of post hoc modeling
Insights from International Baccalaureate multi-language assessments
Bibliographic Data
| ID | 22162124 |
|---|---|
| Authors | Louise Badham (0000-0003-0411-1827, International Baccalaureate, corresponding author), Michelle Meadows (0009-0007-5200-6331, University of Oxford), Jo‐Anne Baird (0000-0001-5974-3237, University of Oxford) |
| Year | 2025 |
| Volume | 10 |
| Publication date | 2025-08-25 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Frontiers in Education (JOURNAL) |
| Journal identifiers | ISSN: 2504-284X • E-ISSN: 2504-284X |
| Publisher | Frontiers Media SA (PUBLISHER • CH) |
| DOI | 10.3389/feduc.2025.1616879 |
| OpenAlex | W4413486553 |
| Language | EN |
| Citations received | 1 |
| References cited | 40 |
Construct comparability was investigated across different subjects in the International Baccalaureate (IB) Diploma Programme (DP). A Rasch Partial Credit Model (PCM) was applied to historical assessment data to generate statistical measures of the relative “difficulty” of IB subjects and languages. Specifically, analysis centered on different language versions of literature assessments, where exams differ in content, but are designed to assess the same target constructs. Rasch analyses were conducted sequentially in three subsets of data. Three different conceptualizations of the linking construct were compared, with the aim of narrowing the definition to increase the validity of the comparisons. These ranged from different DP subjects being linked by “general academic ability,” to linking English, Spanish and Chinese language versions of literature with the more relevant construct of “literary analysis.” Ultimately, the Rasch analyses produced three different rank orders of “difficulty” for the assessments, illustrating the limitations of post hoc construct comparability investigations. Whilst literary analysis is the most theoretically defensible linking construct in this context, the approach relies on bilingual students taking different language versions of the assessments and therefore has limited operational applicability. There are also conceptual limitations, as bilingual examinees are not representative of all students in DP cohorts. Further research is recommended into how cohort characteristics can impact performance, as well as how constructs are defined for use across linguistic and cultural subgroups. Such investigations are crucial to avoid construct bias being introduced in the earliest stages of assessment design
Comparability · Construct validity · Post hoc · Programming language · Psychometrics · Statistics · Computer Science · Mathematics · Medicine · Multilingual Education and Policy · Second Language Learning and Teaching · Student Assessment and Feedback
Adapting Educational and Psychological Tests for Cross-Cultural Assessment
Rasch Analysis in the Human Sciences
Critical Values for Yen’s Q 3
Test translation review
Statistically guided grading judgements
Does item-level DIF manifest itself in scale-level analyses? Implications for translating language tests
QuantCrit
Controversy and the Rasch Model
The Scree Test For The Number Of Factors
Minority language revitalization and educational assessment
| Unique citing works | 1 |
|---|---|
| Citations per year | 1 |
| Citation span | 2025 - 2025 (1) |
| Citation velocity | recent |
| Highly cited | No |
| Citation types | Neutral: 1 |