Utilizing AI questionnaire translations in cross-cultural and intercultural research

Insights and recommendations

Bibliographic Data

ID11208596
AuthorsJonas R Kunst (0000-0002-5319-1256, University of Oslo, corresponding author), Kinga Bierwiaczonek (0000-0002-2251-5633, Iscte – Instituto Universitário de Lisboa)
Year2023
Volume97
Pages101888
Publication date2023-11-01
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueInternational Journal of Intercultural Relations (JOURNAL)
Journal identifiersISSN: 0147-1767 • E-ISSN: 1873-7552
PublisherElsevier BV (PUBLISHER)
DOI10.1016/j.ijintrel.2023.101888
OpenAlexW4386621473
LanguageEN
Citations received11
References cited15

In this research paper, we investigated the viability of AI-supported translations of survey materials in intercultural and cross-cultural research, comparing the quality of machine translations to traditional human translations. Focusing on the HEXACO personality inventory, we translated the original English inventory using Google Translate and GPT-3.5 into 33 languages for which validated human translations exist. The statistical similarity between human- and machine-generated translations varied considerably between the target languages. It was highest for target languages from the same language family as the source language, arguably because this relatedness allowed for more direct machine translations. Consistent with this reasoning, the genetic similarity between languages largely explained the differences observed. GPT’s temperature setting determining how stringently or freely a text is translated had little influence on the similarity estimates, but very high levels tended to produce somewhat lower statistical similarity. To validate the quality of the machine translations, a group of social scientists rated the translation in a language for which the human and machine translations statistically converged strongly. Although the human translation was rated as being of higher quality than four out of five machine translations, these differences were relatively small. Crucially, the social scientists did not rate the human translation as significantly better than the GPT 3.5 translation with the lowest temperature setting. Based on these insights, we propose a framework outlining four recommendations for utilizing AI-supported translation in cross-cultural and intercultural research, involving AI to varying degrees in the forward-back translation process

Biology · Epistemology · Linguistics · Machine translation · Natural language processing · Personality · Quality (philosophy · Similarity (geometry · Translation (biology · Artificial Intelligence · Computational and Text Analysis Methods · Computer Science · Cultural Differences and Values · Psychology · Social Psychology

Tools

Open DOIOpen Access