Seeing Through the Mask

Recognition of Genuine Emotion Through Masked Facial Expression

Dados Bibliográficos

ID22108564
AutoresJu Zhou (0009-0003-3768-635X, Southwest University), Xinyu Liu (0000-0001-5087-3385, Southwest University), Hanpu Wang (0000-0001-7135-3610, Southwest University), Zheyuan Zhang (0009-0005-8086-497X, Southwest University), Tong Chen (0000-0003-3944-957X, Southwest University), Xiaolan Fu (0000-0002-6176-6339, Institute of Psychology, Chinese Academy of Sciences), Guangyuan Liu (0000-0002-8058-5947, Southwest University)
Ano2024
Volume11
Fascículo6
Páginas7159-7172
Data de publicação2024-12-01
Peer ReviewedSim
Open AccessSim
TipoARTICLE
PeriódicoIEEE Transactions on Computational Social Systems (JOURNAL)
Identificadores do periódicoISSN: 2329-924X • E-ISSN: 2373-7476
EditoraInstitute of Electrical and Electronics Engineers (IEEE) (PUBLISHER)
DOI10.1109/tcss.2024.3404611
OpenAlexW4399619448
IdiomaEN
Citações recebidas3
Referências citadas34

The purpose of facial expression recognition is to recognize the corresponding emotions. However, people tend to hide their emotions by displaying facial expressions that differ from those evoked by emotions. These inconsistent facial expressions are referred to as masked facial expressions (MFEs). The automatic recognition of hidden emotions within an MFE using image data is challenging. In this study, we find distinctive movement patterns in the facial action units (AUs) of MFE sequences through a detailed analysis. Considering our findings, we propose handcrafted features called dynamic AU intensity features (DAIFs) to represent AU movement. Furthermore, we develop a decoupled AU transformer (DAUT) model for recognition, where the decoupled convolution operators ensure that the temporal information in the DAIF is not damaged. To further improve the recognition performance, we design self-supervised clip prediction for pretraining of DAUT. Experimental results demonstrate that our proposed method performs exceptionally well across all tasks in the MFE dataset, particularly improving accuracy by nearly double on the most challenging 36-class task. This suggests that leveraging temporal information from facial AU movements is a reliable and effective technique for recognizing MFEs

Cognitive psychology · Computer vision · Facial expression · Facial expression recognition · Facial recognition system · Speech recognition · Color perception and design · Communication · Computer Science · Emotion and Mood Recognition · Psychology · Artificial Intelligence

Ferramentas

Abrir DOI