The impact of nonverbal information on subtitle translators’ cognitive effort
An eye-tracking and key-logging study
Bibliographic Data
| ID | 21359350 |
|---|---|
| Authors | Jie Huang (0009-0007-4279-2914, University of International Business and Economics, Beijing, China, corresponding author) |
| Year | 2024 |
| Volume | 69 |
| Issue | 3 |
| Pages | 784-813 |
| Publication date | 2024-01-01 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Meta Journal des traducteurs (JOURNAL) |
| Journal identifiers | ISSN: 0026-0452 • E-ISSN: 1492-1421 |
| Publisher | Consortium Erudit (PUBLISHER • CA) |
| DOI | 10.7202/1121669ar |
| OpenAlex | W4417043612 |
| Language | EN |
| References cited | 65 |
This study investigates the impact of nonverbal information on the cognitive effort of translators in audiovisual translation (AVT), specifically in subtitling. While the importance of nonverbal elements in AVT is widely recognised, understanding how the amount of nonverbal information directly affects translators’ cognitive effort remains a challenge. To address this, methods from Translation Process Research (TPR) are employed, including eye-tracking, keystroke-logging and subjective reflection. Based on the multimodal transcription method, texts are differentiated according to their level of nonverbal information by analysing key features such as kinesics, camera content and acoustic diegesis. Through the experiment, the cognitive processes and perceived effort of translators are examined throughout the subtitle translation process, including video watching. Results show that an increase in the amount of nonverbal information does not lead to increased cognitive effort among subtitle translators, as evidenced by reduced fixations, keystroke operations and pauses. The perceived effort by translators aligns with this finding. Additionally, the distribution of attention exhibits a consistent pattern, with only minor differences due to varying levels of nonverbal information. Thus, this study highlights the potential to examine the role of nonverbal information in shaping translators’ cognitive processes and confirms the viability of incorporating TPR methods into AVT process research
Cognition · Information processing · Keystroke logging · Nonverbal communication · Process (computing) · Subtitle · Advanced Technology in Applications · Subtitles and Audiovisual Media · Translation Studies and Practices
Perception and communication.
In Other Words
Statistics for Linguists
Cognitive Load Theory
Audiovisual Translation
Mental Representations
Picture superiority in free recall
Working Memory
Translation and mass-communication
Dual coding theory
Defining Audio/Video Redundancy From a Limited- Capacity Information Processing Perspective
The value of teletext sub-titles in language learning
A comparison of conscious and automatic memory processes for picture and word stimuli
Translating Picturebooks
Indicators of difficulty in translation — Correlating product and process data
History, development, challenges and opportunities of empirical research in audiovisual translation
Effects of text chunking on subtitling
Digital resources in the translation process – attention, cognitive effort and processing flow
Post-editing machine translated subtitles
Evaluating the Use of Captioned Video Materials in Advanced Foreign Language Learning
Captioned Videotapes and Second‐Language Listening Word Recognition
Traduire dans un monde de signes non traduits
Yeah, that's it
| Highly cited | No |
|---|