The multimodal facilitation effect in human communication
Bibliographic Data
| ID | 21641369 |
|---|---|
| Authors | Linda Drijvers (0000-0001-9154-7033, Radboud University Nijmegen, corresponding author), Judith Holler (0000-0002-2381-2699, Radboud University Nijmegen) |
| Year | 2023 |
| Volume | 30 |
| Issue | 2 |
| Pages | 792-801 |
| Publication date | 2023-04-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Psychonomic Bulletin & Review (JOURNAL) |
| Journal identifiers | ISSN: 1069-9384 • E-ISSN: 1531-5320 |
| Publisher | Springer Science and Business Media LLC (PUBLISHER) |
| DOI | 10.3758/s13423-022-02178-x |
| PMID | 36138282 |
| OpenAlex | W4296688994 |
| Language | EN |
| Citations received | 4 |
| References cited | 39 |
During face-to-face communication, recipients need to rapidly integrate a plethora of auditory and visual signals. This integration of signals from many different bodily articulators, all offset in time, with the information in the speech stream may either tax the cognitive system, thus slowing down language processing, or may result in multimodal facilitation. Using the classical shadowing paradigm, participants shadowed speech from face-to-face, naturalistic dyadic conversations in an audiovisual context, an audiovisual context without visual speech (e.g., lips), and an audio-only context. Our results provide evidence of a multimodal facilitation effect in human communication: participants were faster in shadowing words when seeing multimodal messages compared with when hearing only audio. Also, the more visual context was present, the fewer shadowing errors were made, and the earlier in time participants shadowed predicted lexical items. We propose that the multimodal facilitation effect may contribute to the ease of fast face-to-face conversational interaction
Cognition · Cognitive psychology · Facilitation · Linguistics · Speech recognition · Communication · Computer Science · Hearing Impairment and Communication · Language, Metaphor, and Cognition · Multisensory perception and integration · Neuroscience · Psychology
When Does it Really Matter? Exploring the Conditions under which Gestures Affect Students’ Language Recall, Pronunciation Accuracy, and Cognitive Load in the EFL classroom
Text availability and the speaker’s visibility in simultaneous interpreting
Multimodal information density is highest in question beginnings, and early entropy is associated with fewer but longer visual signals
Do hand gestures increase perceived prominence in naturally produced utterances
Four central questions about prediction in language processing
Linguistic Structure and Speech Shadowing at Very Short Latencies
Turn-taking in Human Communication – Origins and Implications for Language Processing
The Empathy Quotient
Universals and cultural variation in turn-taking in conversation
Multimodal Language Processing in Human Communication
Two Sides of the Same Coin
BRMS
An experimental study of when and how speakers use gestures to communicate
Interactive gestures
How iconic gestures and speech interact in the representation of meaning
Foreshadowing a problem
| Unique citing works | 4 |
|---|---|
| Citations per year | 2 |
| Citation span | 2024 - 2026 (3) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 4 |