Frontiers in Public Health · 12 Nov 2026
Comparative evaluation of large language model chatbots for pheochromocytoma education: accuracy, information quality, transparency, and readability
Jiadong Zhao, Fuyu Guo, Yang Liu, Peng Yu, Yi Zhang, Qingshan Yang, Shuolei Sun
Citations by database
Each database indexes a different set of journals, preprints and books, so they are shown side by side rather than blended.
- OpenCitations0
- Crossref0
Citations per year
The current year is still counting.
- 2026 so far0
Abstract
Background Large language model (LLM)-based chatbots are increasingly used for medical information, but their suitability for pheochromocytoma education remains uncertain. We compared four contemporary LLMs for knowledge accuracy, information quality, transparency, and readability. Methods ChatGPT (ChatGPT-5.6 Sol), DeepSeek (DeepSeek-V4 Flash), Gemini (Gemini-3.6 Flash), and Grok (Grok-4.5 Fast) answered 200 pheochromocytoma-related multiple-choice questions across three independent trials. Google Trends and clinician input were used to develop 15 patient-centered questions. Responses were evaluated using DISCERN, EQIP, GQS, JAMA benchmark criteria, and six readability indices. Results Overall knowledge accuracy differed among the four models ( p = 0.0012), with mean accuracy rates of 93.00% for ChatGPT, 91.17% for DeepSeek, 90.50% for Gemini, and 81.67% for Grok. However, after adjustment for multiple comparisons, statistically significant pairwise differences were limited to ChatGPT versus Grok for overall accuracy (adjusted p = 0.0129) and clinical-presentation questions (adjusted p = 0.0088). No significant differences were identified among ChatGPT, DeepSeek, and Gemini. Exploratory assessments of 15 patient-oriented questions suggested differences in information quality, transparency, and readability across models. ChatGPT showed comparatively favorable information quality scores, whereas DeepSeek exhibited a more favorable descriptive readability profile. Nevertheless, transparency was limited across all models, and none achieved the recommended sixth-grade readability standard. Conclusion The evaluated LLM-based chatbots demonstrated generally favorable performance on structured pheochromocytoma-related knowledge questions, although statistically significant pairwise differences in accuracy were limited. Findings from the exploratory patient-oriented assessments suggested potential differences in information quality and readability but require confirmation in larger studies. Persistent limitations in transparency and readability highlight the importance of clinician oversight, source verification, and linguistic simplification before these tools are used for patient education.
What the data shows
- No database records a citation yet, as is usual for a paper this new.
Graph
This paper in the middle, its authors and their most-cited papers below, the papers it cites to the left and the papers citing it to the right. The timeline lays every paper out by year.
Select a node for its details. Drag to pan, scroll to zoom, drag a node to move it.
Start with
Cited by 0 papers
Who cites this paper, from open citation data, with Crossref's record of each; the 100 newest when there are more.
References (31)
What it cites, from open citation data, with Crossref's record of each.
Sign in to see 11 more references
A free account opens the rest of the citing papers and references on every paper’s page.