Will I speak louder if I see you struggling to understand? Speech modifications in response to non-verbal visual cues of listening effort

Bibliographic Data

ID21640935
AuthorsElena Gessa (0000-0002-0623-9130, Lyon 1 Université, corresponding author), Chiara Valzolgher (0000-0002-1354-3288, University of Trento), Elena Giovanelli (0000-0001-7753-632X, University of Trento), Tommaso Rosi (0000-0002-7381-6241, University of Trento), Clément Désoche (0000-0003-0932-6747, Lyon 1 Université), Alessandro Farnè (0000-0001-5769-3491, Lyon 1 Université), Francesco Pavani (0000-0003-0426-5578, University of Trento)
Year2026
Volume33
Issue5
Publication date2026-06-01
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenuePsychonomic Bulletin & Review (JOURNAL)
Journal identifiersISSN: 1069-9384 • E-ISSN: 1531-5320
PublisherSpringer Science and Business Media LLC (PUBLISHER)
DOI10.3758/s13423-026-02942-3
PMID42298253
OpenAlexW7164809911
LanguageEN
References cited55

During face-to-face interactions, overt non-verbal cues from conversational partners can reflect the quality of the conversation. Here, we investigated the impact of specific visual non-verbal cues (i.e., facial expressions of confusion, approaching movements towards the speaker), on the speakers’ acoustic–phonetic adaptations typically employed to enhance clarity (i.e., speech rate, voice intensity, and pitch). Participants read sentences to a listener seated in a separate room and visible through a head-mounted display. Crucially, no auditory feedback was provided. Unknown to participants, the listener was a confederate whose behavior was pre-recorded in silence using a 360° video camera. The confederate simulated three different levels of listening effort: easy, medium, and hard. After each trial, participants rated the estimated listening effort and comprehension of the confederate, as well as their own speaking effort. In Experiment 1 ( N = 22), participants were told their voice was transmitted with varying background noise levels, providing a rationale for potential listener difficulties. In Experiment 2 ( N = 12), no mention of background noise was made, to further isolate the effects of visual cues. In both experiments, participants adjusted their speech to the seen listener’s behavior. Experiment 1 showed adaptations across all measured parameters, and significant modulations persisted for most parameters in Experiment 2. These adaptations were already present in the earliest phase of exposure to the visual effort cues and remained stable across the mini-block, suggesting a rapid and highly sensitive monitoring mechanism. These findings indicate that visible non-verbal cues of listening effort can elicit acoustic–phonetic adaptations, even in the absence of acoustic feedback or explicit communicative signals, highlighting their role in regulating communication and maintaining speech clarity

Active listening · CLARITY · Comprehension · Cue-dependent forgetting · Listening comprehension · Sensory cue · Silence · Multisensory perception and integration · Neurobiology of Language and Bilingualism · Phonetics and Phonology Research