Sonification of the self vs. sonification of the other
Differences in the sonification of performed vs. observed simple hand movements
Bibliographic Data
| ID | 21642371 |
|---|---|
| Authors | Roberto Bresin (0000-0002-3086-0322, KTH Royal Institute of Technology, corresponding author), Maurizio Mancini (0000-0002-9933-8583, University College Cork), Ludvig Elblaus (0000-0002-2659-0411, KTH Royal Institute of Technology), Emma Frid (0000-0002-4422-5223, KTH Royal Institute of Technology) |
| Year | 2020 |
| Volume | 144 |
| Pages | 102500 |
| Publication date | 2020-12-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | International Journal of Human-Computer Studies (JOURNAL) |
| Journal identifiers | ISSN: 1071-5819 • E-ISSN: 1095-9300 |
| Publisher | Elsevier BV (PUBLISHER) |
| DOI | 10.1016/j.ijhcs.2020.102500 |
| OpenAlex | W3036207587 |
| Language | EN |
| Citations received | 1 |
| References cited | 10 |
Existing works on interactive sonification of movements, i.e., the translation of human movement qualities from the physical to the auditory domain, usually adopt a predetermined approach: the way in which movement features modulate the characteristics of sound is fixed. In our work we want to go one step further and demonstrate that the user role can influence the tuning of the mapping between movement cues and sound parameters. Here, we aim to verify if and how the mapping changes when the user is either the performer or the observer of a series of body movements (tracing a square or an infinite shape with the hand in the air). We asked participants to tune movement sonification while they were directly performing the sonified movement vs. while watching another person performing the movement and listening to its sonification. Results show that the tuning of the sonification chosen by participants is influenced by three variables: role of the user (performer vs observer), movement quality (the amount of Smoothness and Directness in the movement), and physical parameters of the movements (velocity and acceleration). Performers focused more on the quality of their movement, while observers focused more on the sonic rendering, making it more expressive and more connected to low-level physical features
Accelerometer · Acoustics · Active listening · Auditory display · Auditory feedback · Human–computer interaction · Sonification · Action Observation and Synchronization · Communication · Computer Science · Multisensory perception and integration · Psychology · Tactile and Sensory Interactions · Artificial Intelligence
| Unique citing works | 1 |
|---|---|
| Citations per year | 0,33 |
| Citation span | 2023 - 2023 (1) |
| Citation velocity | historical |
| Highly cited | No |
| Citation types | Neutral: 1 |