Audio-synchronized textual enhancement in foreign language pronunciation learning from videos

Datos Bibliográficos

ID11989645
AutoresValeria Galimberti (0000-0001-6939-0148, Universitat de Barcelona, autor de correspondencia), Joan C Mora (0000-0001-6071-2008, Universitat de Barcelona), Roger Gilabert (0000-0001-6464-4161, Universitat de Barcelona)
Año2023
Volumen116
Páginas103078-103078
Fecha de publicación2023-06-16
Peer ReviewedSí
Open AccessSí
TipoARTICLE
RevistaSystem (JOURNAL)
Identificadores de la revistaISSN: 0346-251X • E-ISSN: 1879-3282
EditorialElsevier BV (PUBLISHER)
DOI10.1016/j.system.2023.103078
OpenAlexW4380926312
IdiomaEN
Citas recibidas6
Referencias citadas30

The benefits of multimodal input on foreign language listening comprehension and vocabulary learning are well-established, but only recently has its impact on pronunciation been explored. In this study, we audio-synchronized the highlighting of target words in captions to promote the activation of their phonolexical representations at the time they are auditorily processed and improve phonological updating in the mental lexicon. We recorded the eye movements of 58 L1-Spanish/Catalan learners of English as they watched two videos with target words (TWs) highlighted 500 ms or 300 ms before auditory onset, highlighted from caption onset or, alternatively, under one of two control conditions (unenhanced and uncaptioned). We assessed updating of phonolexical forms in terms of more accurate and faster rejection of mispronunciations of the TWs from pre-to post-test. Results showed that 300 ms synchronized enhancement and unsynchronized enhancement led to longer fixation duration, unsynchronized enhancement reduced TW skipping probability, and both synchronized conditions promoted higher audio-visual synchrony in learners' caption reading. While only the unsynchronized condition resulted in more accurate responses at post-test, all enhancement conditions led to significantly faster rejection of mispronunciations. These initial findings call for further research on audio-synchronized enhancement and its potential benefits for L2 pronunciation learning

Active listening · Lexicon · Linguistics · Mental lexicon · Natural language processing · Pronunciation · Speech recognition · Vocabulary · Communication · Computer Science · EFL/ESL Teaching and Learning · Psychology · Second Language Acquisition and Learning · Subtitles and Audiovisual Media