An Attention-Based Multimodal Siamese Architecture for Tweet-User Verification
Bibliographic Data
| ID | 22106966 |
|---|---|
| Authors | Chanchal Suman (0000-0003-0204-4835, Indian Institute of Technology Patna), Sriparna Saha (0000-0001-5458-9381, Indian Institute of Technology Patna), Pushpak Bhattacharyya (0000-0001-5319-5508, Indian Institute of Technology Bombay) |
| Year | 2023 |
| Volume | 10 |
| Issue | 5 |
| Pages | 2764-2772 |
| Publication date | 2023-10-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | IEEE Transactions on Computational Social Systems (JOURNAL) |
| Journal identifiers | ISSN: 2329-924X • E-ISSN: 2373-7476 |
| Publisher | Institute of Electrical and Electronics Engineers (IEEE) (PUBLISHER) |
| DOI | 10.1109/tcss.2022.3192909 |
| OpenAlex | W4299430693 |
| Language | EN |
| References cited | 24 |
With the advent of internet technologies, it has created different ways of writing anonymously, which has lead to criminal and malicious activities over social media platforms. Thus, the automatic authentication checking of the available contents is the need of the hour. Social media sites, such as Facebook, Twitter, and so on, are used heavily by the users for sharing of information about their day-to-day activities. The identity of the suspect user is matched against tweets written by the specific user in tweet-user verification process. Writing styles of different users differ from each other, due to unique word choices, emoji selection, sentence formation, and punctuation usage. We have developed a multimodal Siamese-based architecture, which uses attention between the text and emoji parts of the tweet for generating a combined representation for the tweet. Attention helps in selecting the relevant information from different modalities. Modality attention is used for fusing the two modalities (text and emoji). We have used a newly developed multimodal Twitter dataset for evaluating the performance of the proposed model. We achieved an average accuracy, precision, recall, and $F$ -measure values of 68.50%, 78.52%, 69.47%, and 67.05%, respectively. The results show an increase of 2.14% in $F$ -measure in comparison with the current state-of-the-art (SOTA) models for this dataset
Emoji · Hyperlink · Information retrieval · Modalities · Natural language processing · Punctuation · Sentence · Social media · Web page · World Wide Web · Authorship Attribution and Profiling · Computer Science · Spam and Phishing Detection · Topic Modeling · Artificial Intelligence
| Citation velocity | historical |
|---|---|
| Highly cited | No |