Exploring the Role of Artificial Intelligence in Facilitating Assessment of Writing Performance in Second Language Learning

Datos Bibliográficos

ID5900754
AutoresZilu Jiang (0000-0003-1341-4766, Johns Hopkins University, autor de correspondencia), Zexin Xu (The Ohio State University), Zilong Pan (0000-0001-7641-0362, Lehigh University), Jingwen He (0000-0001-6713-4650, Michigan State University), Kui Xie (0000-0002-7173-4859, Michigan State University)
Año2023
Volumen8
Número4
Páginas247
Fecha de publicación2023-10-23
Peer ReviewedSí
Open AccessSí
TipoARTICLE
RevistaLanguages (JOURNAL)
Identificadores de la revistaISSN: 2226-471X • E-ISSN: 2226-471X
EditorialMDPI AG (PUBLISHER • IT)
DOI10.3390/languages8040247
OpenAlexW4387879283
IdiomaEN
Citas recibidas9
Referencias citadas11

This study examined the robustness and efficiency of four large language models (LLMs), GPT-4, GPT-3.5, iFLYTEK and Baidu Cloud, in assessing the writing accuracy of the Chinese language. Writing samples were collected from students in an online high school Chinese language learning program in the US. The official APIs of the LLMs were utilized to conduct analyses at both the T-unit and sentence levels. Performance metrics were employed to evaluate the LLMs' performance. The LLM results were compared to human rating results. Content analysis was conducted to categorize error types and highlight the discrepancies between human and LLM ratings. Additionally, the efficiency of each model was evaluated. The results indicate that GPT models and iFLYTEK achieved similar accuracy scores, with GPT-4 excelling in precision. These findings provide insights into the potential of LLMs in supporting the assessment of writing accuracy for language learners

Categorization · Mathematics education · Natural language processing · Sentence · Computer Science · Natural Language Processing Techniques · Psychology · Second Language Acquisition and Learning · Text Readability and Simplification · Artificial Intelligence