GenAI-enabled versus human assessment of written assignments

Bibliographic Data

ID22423795
AuthorsShuheng Lin (0000-0001-8951-7417, Hong Kong Polytechnic University, corresponding author), Grace Ngai (0000-0002-2027-168X, Hong Kong Polytechnic University), Perry B Y Lee (0000-0003-4290-8546, Hong Kong Polytechnic University), Alison K Y Ip (Hong Kong Polytechnic University), Kam Por Kwan (0000-0002-0823-3744, Hong Kong Polytechnic University)
Year2026
Pages1-15
Publication date2026-07-09
Open AccessYes
TypeARTICLE
VenueInnovations in Education and Teaching International (JOURNAL)
Journal identifiersISSN: 1470-3297 • E-ISSN: 1470-3300
PublisherInforma UK Limited (PUBLISHER • GB)
DOI10.1080/14703297.2026.2699254
OpenAlexW7167791717
LanguageEN
References cited27

This study examines the consistency and agreement between generative AI (GenAI)-enabled and human assessments of written assignments in a teacher development course on service-learning. Using a rubric with nine elements, thirty-seven proposals were independently graded by human assessors and OpenAI GPT-4.1. Results show that GenAI tended to assign higher grades than humans, particularly in elements requiring nuanced judgement, such as civic learning and project feasibility. While overall scores demonstrated slight reliability, agreement at the element level was generally poor, with discrepancies most pronounced in subjective criteria. The study also identified a proportional bias, with GenAI grading more generously at lower and mid-range scores. Findings suggest that zero-shot GenAI grading is unlikely to yield fair or reliable results, prompting the need for future research to identify task-specific conditions under which GenAI assessment may be appropriate and effective.

Evaluation methods · Generative grammar · Higher education · Rubric · Standards-based assessment · Intelligent Tutoring Systems and Adaptive Learning · Online Learning and Analytics · Teaching and Learning Programming

Tools

Open DOIOpen Access