Face2Gesture
Translating Facial Expressions into Robot Movements through Shared Latent Space Neural Networks
Bibliographic Data
| ID | 22190697 |
|---|---|
| Authors | Michael Suguitan (0000-0001-7998-2745, Independent Researcher, New York, USA), Nick DePalma (0000-0003-0531-743X, Plus One Robotics, Pittsburgh, USA), Guy Hoffman (0000-0002-0404-6159, Cornell University), Jessica K Hodgins (0000-0002-1778-883X, Carnegie Mellon University) |
| Year | 2024 |
| Volume | 13 |
| Issue | 3 |
| Pages | 1-18 |
| Publication date | 2024-09-30 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | ACM Transactions on Human-Robot Interaction (JOURNAL) |
| Journal identifiers | ISSN: 2573-9522 • E-ISSN: 2573-9522 |
| Publisher | Association for Computing Machinery (ACM) (PUBLISHER) |
| DOI | 10.1145/3623386 |
| OpenAlex | W4387343928 |
| Language | EN |
| References cited | 33 |
In this work, we present a method for personalizing human-robot interaction by using emotive facial expressions to generate affective robot movements. Movement is an important medium for robots to communicate affective states, but the expertise and time required to craft new robot movements promotes a reliance on fixed preprogrammed behaviors. Enabling robots to respond to multimodal user input with newly generated movements could stave off staleness of interaction and convey a deeper degree of affective understanding than current retrieval-based methods. We use autoencoder neural networks to compress robot movement data and facial expression images into a shared latent embedding space. Then, we use a reconstruction loss to generate movements from these embeddings and triplet loss to align the embeddings by emotion classes rather than data modality. To subjectively evaluate our method, we conducted a user survey and found that generated happy and sad movements could be matched to their source face images. However, angry movements were most often mismatched to sad images. This multimodal data-driven generative method can expand an interactive agent’s behavior library and could be adopted for other multimodal affective applications
Artificial neural network · Autoencoder · Computer vision · Embedding · Emotive · Facial expression · Human–computer interaction · Human–robot interaction · Robot · Computer Science · Emotion and Mood Recognition · Human Pose and Action Recognition · Social Robot Interaction and HRI · Artificial Intelligence
Social Eye Gaze in Human-Robot Interaction
The Zoomorphic Miro Robot’s Affective Expression Design and Perceived Appearance
Modeling the Interactions of Context and Style on Affect in Motion Perception
Gaze behaviour, believability, likability and the iCat
Blossom
A circumplex model of affect
An argument for basic emotions
| Citation velocity | historical |
|---|---|
| Highly cited | No |