Evaluating for Evidence of Sociodemographic Bias in Conversational AI for Mental Health Support
Bibliographic Data
The integration of large language models (LLMs) into healthcare highlights the need to ensure their efficacy while mitigating potential harms, such as the perpetuation of biases. Current evidence on the existence of bias within LLMs remains inconclusive. In this study, we present an approach to investigate the presence of bias within an LLM designed for mental health support. We simulated physician–patient conversations by using a communication loop between an LLM-based conversational agent and digital standardized patients (DSPs) that engaged the agent in dialogue while remaining agnostic to sociodemographic characteristics. In contrast, the conversational agent was made aware of each DSP’s characteristics, including age, sex, race/ethnicity, and annual income. The agent’s responses were analyzed to discern potential systematic biases using the Linguistic Inquiry and Word Count tool. Multivariate regression analysis, trend analysis, and group-based trajectory models were used to quantify potential biases. Among 449 conversations, there was no evidence of bias in both descriptive assessments and multivariable linear regression analyses. Moreover, when evaluating changes in mean tone scores throughout a dialogue, the conversational agent exhibited a capacity to show understanding of the DSPs’ chief complaints and to elevate the tone scores of the DSPs throughout conversations. This finding did not vary by any sociodemographic characteristics of the DSP. Using an objective methodology, our study did not uncover significant evidence of bias within an LLM-enabled mental health conversational agent. These findings offer a complementary approach to examining bias in LLM-based conversational agents for mental health support
Medical emergency · Mental health · Occupational safety and health · Poison control · Psychiatry · Applied Psychology · Artificial Intelligence in Healthcare and Education · Clinical Psychology · Human Factors and Ergonomics · Machine Learning in Healthcare · Medicine · Mental Health via Writing · Psychology
Trends in U.S. Depression Prevalence From 2015 to 2020
ChatGPT Utility in Healthcare Education, Research, and Practice
Self-presentation in medicine
Mental Health Service Utilization Rates Among Commercially Insured Adults in the US During the First Year of the Covid-19 Pandemic
Patient Race/Ethnicity and Quality of Patient–Physician Communication During Medical Visits
Investigating the Relationship between Resident Physician Implicit Bias and Language Use during a Clinical Encounter with Hispanic Patients
Physician Racial Bias and Word Use during Racially Discordant Medical Interactions
Sentiment analysis using Twitter data
| Citation velocity | historical |
|---|---|
| Highly cited | No |