Neural Decoding of Multi-Modal Imagery Behavior Focusing on Temporal Complexity

Bibliographic Data

ID15518699
AuthorsNaoki Furutani (0000-0001-5953-2114, Kanazawa University, corresponding author), Yuta Nariya (The University of Tokyo), Tetsuya Takahashi (0000-0001-6068-7623, Kanazawa University, corresponding author), Haruka Ito (0009-0003-8100-5873), Yuko Yoshimura (0000-0001-9226-4561, Kanazawa University), Hirotoshi Hiraishi (0000-0002-6356-6111, Hamamatsu University School of Medicine), Chiaki Hasegawa (0000-0001-8730-8769, Kanazawa University), Takashi Ikeda (0000-0001-9550-5033, Kanazawa University), Mitsuru Kikuchi (0000-0003-3013-7844, Kanazawa University)
Year2020
Volume11
Pages746-746
Publication date2020-07-30
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueFrontiers in Psychiatry (JOURNAL)
Journal identifiersISSN: 1664-0640 • E-ISSN: 1664-0640
PublisherFrontiers Media (PUBLISHER • CH)
DOI10.3389/fpsyt.2020.00746
PMID32848924
OpenAlexW3046279878
LanguageEN
Citations received1
References cited87

Mental imagery behaviors of various modalities include visual, auditory, and motor behaviors. Their alterations are pathologically involved in various psychiatric disorders. Results of earlier studies suggest that imagery behaviors are correlated with the modulated activities of the respective modality-specific regions and the additional activities of supramodal imagery-related regions. Additionally, despite the availability of complexity analysis in the neuroimaging field, it has not been used for neural decoding approaches. Therefore, we sought to characterize neural oscillation related to multimodal imagery through complexity-based neural decoding. For this study, we modified existing complexity measures to characterize the time evolution of temporal complexity. We took magnetoencephalography (MEG) data of eight healthy subjects as they performed multimodal imagery and non-imagery tasks. The MEG data were decomposed into amplitude and phase of sub-band frequencies by Hilbert-Huang transform. Subsequently, we calculated the complexity values of each reconstructed time series, along with raw data and band power for comparison, and applied these results as inputs to decode visual perception (VP), visual imagery (VI), motor execution (ME), and motor imagery (MI) functions. Consequently, intra-subject decoding with the complexity yielded a characteristic sensitivity map for each task with high decoding accuracy. The map is inverted in the occipital regions between VP and VI and in the central regions between ME and MI. Additionally, replacement of the labels into two classes as imagery and non-imagery also yielded better classification performance and characteristic sensitivity with the complexity. It is particularly interesting that some subjects showed characteristic sensitivities not only in modality-specific regions, but also in supramodal regions. These analyses indicate that two-class and four-class classifications each provided better performance when using complexity than when using raw data or band power as input. When inter-subject decoding was used with the same model, characteristic sensitivity maps were also obtained, although their decoding performance was lower. Results of this study underscore the availability of complexity measures in neural decoding approaches and suggest the possibility of a modality-independent imagery-related mechanism. The use of time evolution of temporal complexity in neural decoding might extend our knowledge of the neural bases of hierarchical functions in the human brain

Algorithm · Brain–computer interface · Cognition · Decoding methods · Electroencephalography · Magnetoencephalography · Mental image · Modalities · Modality (human–computer interaction · Motor imagery · Neural decoding · Neuroimaging · Pattern recognition (psychology · Computer Science · EEG and Brain-Computer Interfaces · Neural and Behavioral Psychology Studies · Neural dynamics and brain function · Neuroscience · Psychology · Artificial Intelligence

Tools

Open DOIOpen Access