Exploring the Role of Artificial Intelligence in Facilitating Assessment of Writing Performance in Second Language Learning

Dados Bibliográficos

ID5900754
AutoresZilu Jiang (0000-0003-1341-4766, Johns Hopkins University, autor correspondente), Zexin Xu (The Ohio State University), Zilong Pan (0000-0001-7641-0362, Lehigh University), Jingwen He (0000-0001-6713-4650, Michigan State University), Kui Xie (0000-0002-7173-4859, Michigan State University)
Ano2023
Volume8
Fascículo4
Páginas247
Data de publicação2023-10-23
Peer ReviewedSim
Open AccessSim
TipoARTICLE
PeriódicoLanguages (JOURNAL)
Identificadores do periódicoISSN: 2226-471X • E-ISSN: 2226-471X
EditoraMDPI AG (PUBLISHER • IT)
DOI10.3390/languages8040247
OpenAlexW4387879283
IdiomaEN
Citações recebidas9
Referências citadas11

This study examined the robustness and efficiency of four large language models (LLMs), GPT-4, GPT-3.5, iFLYTEK and Baidu Cloud, in assessing the writing accuracy of the Chinese language. Writing samples were collected from students in an online high school Chinese language learning program in the US. The official APIs of the LLMs were utilized to conduct analyses at both the T-unit and sentence levels. Performance metrics were employed to evaluate the LLMs' performance. The LLM results were compared to human rating results. Content analysis was conducted to categorize error types and highlight the discrepancies between human and LLM ratings. Additionally, the efficiency of each model was evaluated. The results indicate that GPT models and iFLYTEK achieved similar accuracy scores, with GPT-4 excelling in precision. These findings provide insights into the potential of LLMs in supporting the assessment of writing accuracy for language learners

Categorization · Mathematics education · Natural language processing · Sentence · Computer Science · Natural Language Processing Techniques · Psychology · Second Language Acquisition and Learning · Text Readability and Simplification · Artificial Intelligence