A Multimodal Framework for Depression Detection During Covid-19 via Harvesting Social Media

The recent coronavirus disease (COVID-19) has become a pandemic and has affected the entire globe. During the pandemic, we have observed a spike in cases related to mental health, such as anxiety, stress, and depression. Depression significantly influences most diseases worldwide, making it difficult to detect mental health conditions in people due to unawareness and unwillingness to consult a doctor. However, nowadays, people extensively use online social media platforms to express their emotions and thoughts. Hence, social media platforms are now becoming a large data source that can be utilized for detecting depression and mental illness. However, the existing approaches often overlook data sparsity in tweets and the multimodal aspects of social media. In this article, we propose a novel multimodal framework that combines textual, user-specific, and image analysis to detect depression among social media users. To provide enough context about the user’s emotional state, we propose the following: 1) an extrinsic feature by harnessing the URLs present in tweets and 2) extracting textual content present in images posted in tweets. We also extract five sets of features belonging to different modalities to describe a user. In addition, we introduce a deep learning model, the visual neural network (VNN), to generate embeddings of user-posted images, which are used to create the visual feature vector for prediction. We contribute a curated COVID-19 dataset of depressed and nondepressed users for research purposes and demonstrate the effectiveness of our model in detecting depression during the COVID-19 outbreak. Our model outperforms the existing state-of-the-art methods over a benchmark dataset by 2%–8% and produces promising results on the COVID-19 dataset. Our analysis highlights the impact of each modality and provides valuable insights into users’ mental and emotional states

Anxiety · Data science · Disease · Internet privacy · Machine learning · Mental health · Modalities · Pandemic · Psychiatry · Sentiment analysis · Social media · Sociology · Topic model · Visualization · World Wide Web · Computer Science · Digital Mental Health Interventions · Medicine · Mental Health via Writing · Psychology · Sentiment Analysis and Opinion Mining · Artificial Intelligence

• Advancements in Affective Disorder Detection

  Open Access•Fuze Tian, Lixin Zhang et al.•IEEE Transactions on Computational…•2024

  Incluir item

• Depression detection in social media posts using transformer-based models and auxiliary features

  Marios Kerasiotis, Loukas Ilias et al.

  Incluir item

• Content Analysis

  Klaus Krippendorff•Content Analysis•2019

  Incluir item

• Mental Health, Risk Factors, and Social Media Use During the Covid-19 Epidemic and Cordon Sanitaire Among the Community and Health Professionals in Wuhan, China

  Open Access•Michael Y Ni, Lin Yang et al.•JMIR Mental Health•2020

  Incluir item

• Sentiment of Emojis

  Open Access•Petra Kralj Novak, Jasmina Smailović et al.•PLoS ONE•2015

  Incluir item

• Generalized anxiety disorder, depressive symptoms and sleep quality during Covid-19 outbreak in China

  Open Access•Yeen Huang, Ning Zhao•Psychiatry Research•2020

  Incluir item

• Early depression detection in social media based on deep learning and underlying emotions

  Open Access•José Solenir L Figuerêdo, Ana Lúcia L Marreiros Maia et al.•Online Social Networks and Media•2022

  Incluir item

• Detecting Community Depression Dynamics Due to Covid-19 Pandemic in Australia

  Open Access•Jianlong Zhou, Hamad Zogan et al.•IEEE Transactions on Computational…•2021

  Incluir item

• Hashtag recommendation for enhancing the popularity of social media posts

  Open Access•Purnadip Chakrabarti, Eish Malvi et al.•Social Network Analysis and Mining•2023

  Incluir item