Assessment of vaccine information accuracy across large language models
Bibliographic Data
| ID | 22085530 |
|---|---|
| Authors | Julia Siebielec (Medical University of Warsaw, corresponding author), Filip Raciborski (0000-0003-0562-0260, Medical University of Warsaw) |
| Year | 2026 |
| Volume | 14 |
| Pages | 1811462-1811462 |
| Publication date | 2026-06-17 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Frontiers in Public Health (JOURNAL) |
| Journal identifiers | ISSN: 2296-2565 • E-ISSN: 2296-2565 |
| Publisher | Frontiers Media SA (PUBLISHER • CH) |
| DOI | 10.3389/fpubh.2026.1811462 |
| PMID | 42388752 |
| OpenAlex | W7165022696 |
| Language | EN |
| References cited | 37 |
Background: Use of Large Language Models (LLMs) as a health information source is on the rise. As the amount of people willing to self-diagnose with the tools' help is increasing, verifying the correctness of generated responses becomes extremely important. Methods: Four LLMs (ChatGPT, Copilot, DeepSeek, Gemini) were chosen for the analysis. All models were asked 90 multiple-choice questions on the various topics surrounding vaccination, sourced from a government information website. Additionally, the LLMs were asked two open-ended questions regarding the vaccine information sources, as well as advantages and disadvantages of universal vaccination. Answers were analysed quantitatively and qualitatively. Data was collected in March 2026. Results: = 0.0049). While answering open-ended questions, all LLMs were able to cite various, appropriate vaccine information sources. For advantages of vaccination all models pointed to herd immunity. Answers regarding disadvantages to vaccination all included the possibility of adverse reactions. The answers were judged to be factually correct, yet lacking in detail. Conclusion: Tested LLMs were able to identify correct option when presented with multiple choices and to prepare correct, yet vague information when asked an open-ended question. This points to a limited potential of LLMs serving as a vaccine information source, as the generated content, while factually correct, was not enough to be usable on its own
Closeness · Correctness · Herd immunity · USable · Vaccination · Artificial Intelligence in Healthcare and Education · Immune responses and vaccinations · Vaccine Coverage and Hesitancy
Medical knowledge of ChatGPT in public health, infectious diseases, Covid-19 pandemic, and vaccines
Decoupled quality and readability in skin cancer education from large language models
The information challenge in public health crises
How generative artificial intelligence portrays science
HPV Vaccination
When AI models take the exam
| Citation velocity | historical |
|---|---|
| Highly cited | No |