Seeing Through the Mask

Recognition of Genuine Emotion Through Masked Facial Expression

Datos Bibliográficos

ID22108564
AutoresJu Zhou (0009-0003-3768-635X, Southwest University), Xinyu Liu (0000-0001-5087-3385, Southwest University), Hanpu Wang (0000-0001-7135-3610, Southwest University), Zheyuan Zhang (0009-0005-8086-497X, Southwest University), Tong Chen (0000-0003-3944-957X, Southwest University), Xiaolan Fu (0000-0002-6176-6339, Institute of Psychology, Chinese Academy of Sciences), Guangyuan Liu (0000-0002-8058-5947, Southwest University)
Año2024
Volumen11
Número6
Páginas7159-7172
Fecha de publicación2024-12-01
Peer ReviewedSí
Open AccessSí
TipoARTICLE
RevistaIEEE Transactions on Computational Social Systems (JOURNAL)
Identificadores de la revistaISSN: 2329-924X • E-ISSN: 2373-7476
EditorialInstitute of Electrical and Electronics Engineers (IEEE) (PUBLISHER)
DOI10.1109/tcss.2024.3404611
OpenAlexW4399619448
IdiomaEN
Citas recibidas3
Referencias citadas34

The purpose of facial expression recognition is to recognize the corresponding emotions. However, people tend to hide their emotions by displaying facial expressions that differ from those evoked by emotions. These inconsistent facial expressions are referred to as masked facial expressions (MFEs). The automatic recognition of hidden emotions within an MFE using image data is challenging. In this study, we find distinctive movement patterns in the facial action units (AUs) of MFE sequences through a detailed analysis. Considering our findings, we propose handcrafted features called dynamic AU intensity features (DAIFs) to represent AU movement. Furthermore, we develop a decoupled AU transformer (DAUT) model for recognition, where the decoupled convolution operators ensure that the temporal information in the DAIF is not damaged. To further improve the recognition performance, we design self-supervised clip prediction for pretraining of DAUT. Experimental results demonstrate that our proposed method performs exceptionally well across all tasks in the MFE dataset, particularly improving accuracy by nearly double on the most challenging 36-class task. This suggests that leveraging temporal information from facial AU movements is a reliable and effective technique for recognizing MFEs

Cognitive psychology · Computer vision · Facial expression · Facial expression recognition · Facial recognition system · Speech recognition · Color perception and design · Communication · Computer Science · Emotion and Mood Recognition · Psychology · Artificial Intelligence

Herramientas

Abrir DOI