Seeing Through the Mask
Recognition of Genuine Emotion Through Masked Facial Expression
Bibliographic Data
| ID | 22108564 |
|---|---|
| Authors | Ju Zhou (0009-0003-3768-635X, Southwest University), Xinyu Liu (0000-0001-5087-3385, Southwest University), Hanpu Wang (0000-0001-7135-3610, Southwest University), Zheyuan Zhang (0009-0005-8086-497X, Southwest University), Tong Chen (0000-0003-3944-957X, Southwest University), Xiaolan Fu (0000-0002-6176-6339, Institute of Psychology, Chinese Academy of Sciences), Guangyuan Liu (0000-0002-8058-5947, Southwest University) |
| Year | 2024 |
| Volume | 11 |
| Issue | 6 |
| Pages | 7159-7172 |
| Publication date | 2024-12-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | IEEE Transactions on Computational Social Systems (JOURNAL) |
| Journal identifiers | ISSN: 2329-924X • E-ISSN: 2373-7476 |
| Publisher | Institute of Electrical and Electronics Engineers (IEEE) (PUBLISHER) |
| DOI | 10.1109/tcss.2024.3404611 |
| OpenAlex | W4399619448 |
| Language | EN |
| Citations received | 3 |
| References cited | 34 |
The purpose of facial expression recognition is to recognize the corresponding emotions. However, people tend to hide their emotions by displaying facial expressions that differ from those evoked by emotions. These inconsistent facial expressions are referred to as masked facial expressions (MFEs). The automatic recognition of hidden emotions within an MFE using image data is challenging. In this study, we find distinctive movement patterns in the facial action units (AUs) of MFE sequences through a detailed analysis. Considering our findings, we propose handcrafted features called dynamic AU intensity features (DAIFs) to represent AU movement. Furthermore, we develop a decoupled AU transformer (DAUT) model for recognition, where the decoupled convolution operators ensure that the temporal information in the DAIF is not damaged. To further improve the recognition performance, we design self-supervised clip prediction for pretraining of DAUT. Experimental results demonstrate that our proposed method performs exceptionally well across all tasks in the MFE dataset, particularly improving accuracy by nearly double on the most challenging 36-class task. This suggests that leveraging temporal information from facial AU movements is a reliable and effective technique for recognizing MFEs
Cognitive psychology · Computer vision · Facial expression · Facial expression recognition · Facial recognition system · Speech recognition · Color perception and design · Communication · Computer Science · Emotion and Mood Recognition · Psychology · Artificial Intelligence
Principal Components Analysis
The Expression Of The Emotions In Man And Animals
Facial Action Coding System
Common Latent Embedding Space for Cross-Domain Facial Expression Recognition
Toward Artificial Emotional Intelligence for Cooperative Social Human–Machine Interaction
Transfer Model Collaborating Metric Learning and Dictionary Learning for Cross-Domain Facial Expression Recognition
| Unique citing works | 3 |
|---|---|
| Citations per year | 3 |
| Citation span | 2025 - 2026 (2) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 3 |