Decoupled Multimodal Fusion Network Based on Peripheral Physiological Signals
Bibliographic Data
| ID | 22107808 |
|---|---|
| Authors | Tianqi Fan (0000-0003-1750-3221, Beijing Institute of Technology), Sen Qiu (0000-0001-6846-546X, Dalian University of Technology), Yongzhen Wang (0000-0001-6002-4838, Chinese People's Liberation Army), Zhelong Wang (0000-0003-4959-3372, Dalian University of Technology), Hongyu Zhao (0009-0003-1925-3403, Dalian University of Technology), Junhan Jiang (China Medical University), Junnan Xu (0000-0002-1269-1537, Liaoning Cancer Hospital & Institute), Tao Sun (0000-0001-5509-0988, Liaoning Cancer Hospital & Institute), Fuze Tian (0000-0001-6734-0585, Lanzhou University) |
| Year | 2026 |
| Volume | 13 |
| Issue | 3 |
| Pages | 2969-2981 |
| Publication date | 2026-06-01 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | IEEE Transactions on Computational Social Systems (JOURNAL) |
| Journal identifiers | ISSN: 2329-924X • E-ISSN: 2373-7476 |
| Publisher | Institute of Electrical and Electronics Engineers (IEEE) (PUBLISHER) |
| DOI | 10.1109/tcss.2026.3668039 |
| OpenAlex | W7133650238 |
| Language | EN |
| References cited | 55 |
Multimodal peripheral physiological signal fusion for emotion recognition seeks to perceive or recognize human emotions using peripheral modalities such as electromyography, electrodermal activity, and respiratory wave. Previous approaches to multimodal fusion primarily focus on emotion-sensitive signals such as electroencephalogram (EEG), often overlooking the potential value of peripheral physiological signals in emotion recognition. Moreover, the inherent heterogeneity among different modalities continues to pose challenges to fusion quality. In this article, we propose a multimodal decoupled multimodal fusion (DMF) to address these issues, enabling flexible feature decoupling, cross-modal feature interaction, and relational knowledge learning. Specifically, each modality’s representation is first decoupled into two components: common features and modality-specific features; second, the DMF employs progressive cross attention to facilitate the exchange of modality-specific features across different modalities; and finally, it uses relational knowledge to learn multimodal spliced features, embedding both inter-modal and intra-modal feature relationships. DMF offers a dynamic multimodal emotion recognition framework that leverages the emotional information contained in diverse modalities. Experimental results demonstrate that the DMF method consistently outperforms previous approaches and provides a viable solution for multimodal peripheral physiological signal fusion
Artificial neural network · Fusion · Peripheral · Sensor Fusion · EEG and Brain-Computer Interfaces · Emotion and Mood Recognition · Non-Invasive Vital Sign Monitoring · Signal Processing
| Highly cited | No |
|---|