Sonic Sleight of Hand
Sound Induces Illusory Distortions in the Perception and Prediction of Robot Action
Bibliographic Data
| ID | 21199576 |
|---|---|
| Authors | Joel Currie (0000-0002-3367-7056, University of Aberdeen, corresponding author), Maria Elena Giannaccini (0000-0002-0871-4804, University of Aberdeen), Patric Bach (0000-0003-4493-2080, University of Aberdeen) |
| Year | 2025 |
| Volume | 17 |
| Issue | 10 |
| Pages | 1957-1975 |
| Publication date | 2025-10-01 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | International Journal of Social Robotics (JOURNAL) |
| Journal identifiers | ISSN: 1875-4791 • E-ISSN: 1875-4805 |
| Publisher | Springer Science and Business Media LLC (PUBLISHER) |
| DOI | 10.1007/s12369-024-01105-5 |
| PMID | 41169785 |
| OpenAlex | W4391913753 |
| Language | EN |
| References cited | 86 |
For efficient human–robot interaction, human operators need to be able to efficiently represent the robot’s movements in space and predict its next steps. However, according to frameworks of Bayesian multisensory integration, features outside the motion itself—like the sounds a robot makes while it moves—should affect how otherwise identical motions are perceived. Here, we translate an established psychophysical task from experimental psychology to a human–robot interaction context, which can measure these distortions to motion perception. In two series of preregistered studies, participants watched a humanoid robot make forward and backward reaching movements. When the robot hand suddenly disappeared, they reported its last seen location, either with the mouse cursor (Experiment 1a and 1b) or by matching it to probe stimuli in different locations (Experiment 2a and 2b). The results revealed that even small changes to the robot’s sound robustly affect participants’ visuospatial representation of its motions, so that the motion appeared to extend further in space when accompanied by slightly (100 ms) longer sounds compared to slightly shorter sounds (100 ms shorter). Moreover, these sound changes do not only affect where people currently locate the robot’s motion, but where they anticipate its future steps. These findings show that sound design is an effective medium for manipulating how people represent otherwise identical robot actions and coordinate its interactions with it. The study acts as proof of concept that psychophysical tasks provide a promising tool to measure how design parameters influence the perception and prediction of robot motion
Computer vision · Human–computer interaction · Humanoid robot · Perception · Robot · Robotics · Computer Science · Emotion and Mood Recognition · Multisensory perception and integration · Psychology · Social Robot Interaction and HRI · Artificial Intelligence
The Ventriloquist Effect Results from Near-Optimal Bimodal Integration
Serial dependence in visual perception
Merging the senses into a robust percept
Hearing lips and seeing voices
The thing that should not be
Joint action
Reducing consistency in human realism increases the uncanny valley effect; increasing category uncertainty does not
A Mass-Produced Sociable Humanoid Robot
The Bayesian brain
Sounding Robots
“I See What You Did There”
Nonverbal Sound in Human-Robot Interaction
Addressing joint action challenges in HRI
Humans are Well Tuned to Detecting Agents Among Non-agents
On the Imitation of Goal Directed Movements of a Humanoid Robot
Biological movement increases acceptance of humanoid robots as human partners in motor interaction
| Highly cited | No |
|---|