An Evaluation of ChatGPT's Translation Accuracy Using BLEU Score
Bibliographic Data
| ID | 21807498 |
|---|---|
| Authors | Mozhgan Ghassemiazghandi (0000-0002-3038-3124, Hospital Universiti Sains Malaysia, corresponding author) |
| Year | 2024 |
| Volume | 14 |
| Issue | 4 |
| Pages | 985-994 |
| Publication date | 2024-04-29 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Theory and Practice in Language Studies (JOURNAL) |
| Journal identifiers | ISSN: 1799-2591 • E-ISSN: 2053-0692 |
| Publisher | Academy Publication (PUBLISHER • GB) |
| DOI | 10.17507/tpls.1404.07 |
| OpenAlex | W4396496351 |
| Language | EN |
| Citations received | 3 |
| References cited | 4 |
Traditional views have long held that machine translation cannot achieve the quality and accuracy of human translators, especially in complex language pairs like Persian and English. This study challenges this perspective by demonstrating that ChatGPT-4, with access to vast amounts of multilingual data and leveraging advanced large language model algorithms, significantly outperforms widely utilized open-source machine translation tools and approaches the realm of human translation quality. This research aims to critically assess the translation accuracy of ChatGPT-4 against a traditional open-source machine translation tool from Persian to English, highlighting the advancements in artificial intelligence-driven translation technologies. Using Bilingual Evaluation Understudy scores for a comprehensive evaluation, this study compares the translation outputs from ChatGPT-4 with MateCat, providing a quantitative basis for comparing their accuracy and quality. ChatGPT-4 achieves a BLUE score of 0.88 and an accuracy of 0.68, demonstrating superior performance compared to MateCat, with a 0.82 BLUE score and 0.49 accuracy. The results indicate that the translations generated by ChatGPT-4 surpass those produced by MateCat and nearly mirror the quality of human translations. The evaluation demonstrates the effectiveness of OpenAI's large language model algorithms in improving translation accuracy
BLEU · Machine translation · Natural language processing · Artificial Intelligence in Healthcare and Education · Chemistry · Computer Science · Artificial Intelligence
| Citations per year | 3 |
|---|---|
| Citation span | 2025 - 2026 (2) |
| Highly cited | No |