Calibration of Transformer-Based Models for Identifying Stress and Depression in Social Media

Bibliographic Data

ID22108515
AuthorsLoukas Ilias (0000-0002-4483-4264, National Technical University of Athens), Spiros Mouzakitis (0000-0001-9616-447X, National Technical University of Athens), Dimitris Askounis (0000-0002-2618-5715, National Technical University of Athens)
Year2024
Volume11
Issue2
Pages1979-1990
Publication date2024-04-01
Open AccessYes
TypeARTICLE
VenueIEEE Transactions on Computational Social Systems (JOURNAL)
Journal identifiersISSN: 2329-924X • E-ISSN: 2373-7476
PublisherInstitute of Electrical and Electronics Engineers (IEEE) (PUBLISHER)
DOI10.1109/tcss.2023.3283009
OpenAlexW4380881406
LanguageEN
Citations received8
References cited59

In today’s fast-paced world, the rates of stress and depression present a surge. People use social media for expressing their thoughts and feelings through posts. Therefore, social media provide assistance for the early detection of mental health conditions. Existing methods mainly introduce feature extraction approaches and train shallow machine learning (ML) classifiers. For addressing the need of creating a large feature set and obtaining better performance, other research studies use deep neural networks or language models based on transformers. Despite the fact that transformer-based models achieve noticeable improvements, they cannot often capture rich factual knowledge. Although there have been proposed a number of studies aiming to enhance the pretrained transformer-based models with extra information or additional modalities, no prior work has exploited these modifications for detecting stress and depression through social media. In addition, although the reliability of a machine learning (ML) model’s confidence in its predictions is critical for high-risk applications, there is no prior work taken into consideration the model calibration. To resolve the above issues, we present the first study in the task of depression and stress detection in social media, which injects extra-linguistic information in transformer-based models, namely, bidirectional encoder representations from transformers (BERT) and MentalBERT. Specifically, the proposed approach employs a multimodal adaptation gate for creating the combined embeddings, which are given as input to a BERT (or MentalBERT) model. For taking into account the model calibration, we apply label smoothing. We test our proposed approaches in three publicly available datasets and demonstrate that the integration of linguistic features into transformer-based models presents a surge in performance. Also, the usage of label smoothing contributes to both the improvement of the model’s performance and the calibration of the model. We finally perform a linguistic analysis of the posts and show differences in language between stressful and nonstressful texts, as well as depressive and nondepressive posts

Calibration · Electrical engineering · Reliability engineering · Statistics · Transformer · Voltage · Computer Science · Digital Mental Health Interventions · Engineering · Mathematics · Mental Health via Writing · Psychology · Sentiment Analysis and Opinion Mining

Tools

Open DOIOpen Access