Multimodal Knowledge Graph Embedding With Missing Data Integration

In real-world network scenarios, modal absence may be caused by various factors, such as sensor damage, data corruption, and human errors in recording. Effectively integrating multimodal missing data still poses significant challenges. Different combinations of missing modes can form feature sets of inconsistent dimensions and quantities. Additionally, effectively merging multimodal data requires a thorough understanding of specific modal information and intermodal interactions. The abundance of missing data can significantly reduce the sample set size, leading to learning interaction features from only a few samples. Moreover, there is a lack of clear correspondence between heterogeneous data from different sources. To address these issues, we focus our research on multimodal knowledge graph scenarios with different types of structures and content and develop a new knowledge graph embedding method. First, we use three embedding components to automatically extract feature vector representations of items from the structural content, textual content, and visual content of the knowledge graph. Then, we divide the dataset into several modal groups and model these modal groups using a multilayer network structure, with each multilayer network corresponding to a specific multimodal combination. Subsequently, we construct corresponding multilayer network projection layers and propose a two-stage GAT-based transfer learning framework for the projection layers, in which the extracted incomplete multimodal information and intermodal interaction information are integrated and mapped to a low-dimensional space. Finally, we not only theoretically prove the feasibility of the proposed method but also validate its effectiveness through extensive comparative experiments on multiple datasets

Data integration · Data mining · Embedding · Graph · Graph theory · Knowledge graph · Advanced Graph Neural Networks · Cognitive Computing and Networks · Computer Science · Mathematics · Semantic Web and Ontologies · Artificial Intelligence · Theoretical Computer Science

• MLFormer

  Open Access•Meng Wang, Changyu Li et al.•IEEE Transactions on Computational…•2025

  Incluir item