AI-Assisted Exam Variant Generation

A Human-in-the-Loop Framework for Automatic Item Creation

Bibliographic Data

ID22044568
AuthorsCharles MacDonald Burke (0009-0003-5510-7809, Franklin University Switzerland, corresponding author)
Year2025
Volume15
Issue8
Pages1029
Publication date2025-08-11
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueEducation Sciences (JOURNAL)
Journal identifiersISSN: 2227-7102 • E-ISSN: 2227-7102
PublisherMDPI AG (PUBLISHER • IT)
DOI10.3390/educsci15081029
OpenAlexW4413215135
LanguageEN
Citations received2
References cited20

Educational assessment relies on well-constructed test items to measure student learning accurately, yet traditional item development is time-consuming and demands specialized psychometric expertise. Automatic item generation (AIG) offers template-based scalability, and recent large language model (LLM) advances promise to democratize item creation. However, fully automated approaches risk introducing factual errors, bias, and uneven difficulty. To address these challenges, we propose and evaluate a hybrid human-in-the-loop (HITL) framework for AIG that combines psychometric rigor with the linguistic flexibility of LLMs. In a Spring 2025 case study at Franklin University Switzerland, the instructor collaborated with ChatGPT (o4-mini-high) to generate parallel exam variants for two undergraduate business courses: Quantitative Reasoning and Data Mining. The instructor began by defining “radical” and “incidental” parameters to guide the model. Through iterative cycles of prompt, review, and refinement, the instructor validated content accuracy, calibrated difficulty, and mitigated bias. All interactions (including prompt templates, AI outputs, and human edits) were systematically documented, creating a transparent audit trail. Our findings demonstrate that a HITL approach to AIG can produce diverse, psychometrically equivalent exam forms with reduced development time, while preserving item validity and fairness, and potentially reducing cheating. This offers a replicable pathway for harnessing LLMs in educational measurement without sacrificing quality, equity, or accountability

Accountability · Audit · Data science · Database · Human-in-the-loop · Scalability · Workflow · Artificial Intelligence in Healthcare and Education · Computer Science · Intelligent Tutoring Systems and Adaptive Learning · Online Learning and Analytics · Artificial Intelligence

Tools

Open DOIOpen Access