The Relationship Between Surprisal and Prosodic Prominence in Conversation Reflects Intelligibility‐Oriented Pressures
Bibliographic Data
Conversation is a dynamic, multimodal activity involving the exchange of complex streams of information like words, prosody, gesture, eye contact, and backchannels. Understanding how these different channels interact in naturalistic scenarios is essential for understanding the mechanisms governing human communication. Past studies suggested that the duration of words is tied to their predictability in context, but it remains unclear whether this relationship is speaker‐oriented (e.g., retrieval or production‐based) or due to listener‐oriented, intelligibility‐based pressures (i.e., emphasizing unpredictable words to ease comprehension). This study aims to examine the relationship between predictability and additional acoustic variables, to test how much intelligibility‐oriented principles impact conversation. We use the GPT‐2 large language model to assess the relationship between surprisal, a measure of unpredictability, and several variables known to play an important role in conversation—the prosodic features of duration, intensity, and pitch. We perform this analysis on the CANDOR corpus of naturalistic spoken video call conversation between strangers in English. In keeping with previous results using n‐gram predictability, we find that GPT‐2 surprisal predicts significantly higher values for duration. Moreover, surprisal also predicts maximum pitch and pitch range even when controlling for duration, with mixed evidence for an effect of surprisal on intensity. Additionally, we investigated listener backchannels (short interjections like “yeah” or “mhm”) and found that listener backchannels tended to be accompanied and followed by a spike in the surprisal of speakers' words. Finally, we demonstrate a divergence between the effect of context window size on the model fit of surprisal to maximum pitch and to other variables. The results provide additional support for intelligibility‐based accounts, which hold that language production is sensitive to a pressure for successful communication, not just speaker‐oriented pressures. Our data and analysis code are shared: https://osf.io/sqpn6/?view_only=e4d9e36c68b54863bc781e359463e1fe
Conversation · Conversation analysis · Predictability · Turn-taking · Hearing Impairment and Communication · Multisensory perception and integration · Phonetics and Phonology Research
The effect of word predictability on reading time is logarithmic
Probabilistic relations between words
Language redundancy predicts syllabic duration and the spectral characteristics of vocalic syllable nuclei
Redundancy and reduction
Why reduce? Phonological neighborhood density and phonetic reduction in spontaneous speech
Switchboard
Moving beyond Kučera and Francis
Experimental and theoretical advances in prosody
Predictability effects on durations of content and function words in conversational English
Montreal Forced Aligner
Word informativity influences acoustic duration
Signal Smoothing and Syntactic Choices
Prosody leaks into the memories of words
The role of constructions in understanding predictability measures and their correspondence to word duration
Affirmative Cue Words in Task-Oriented Dialogue
Prosodic features which cue back-channel responses in English and Japanese
The Smooth Signal Redundancy Hypothesis
Introducing Parselmouth
Communication Strategies in a Second Language
Addressee backchannels steer narrative development
(Don't) try this at home! The effects of recording devices and software on phonetic analysis
Multimodel Inference
| Highly cited | No |
|---|