Frontiers in Digital Health · 11 Nov 2026
Multidimensional evaluation of large language models for patient education on asthma during pregnancy: a cross-sectional comparative study
Zhoujun Zhu, Cong Xie, Haiyan Wu
Citations by database
Each database indexes a different set of journals, preprints and books, so they are shown side by side rather than blended.
- OpenCitations0
- Crossref0
Citations per year
The current year is still counting.
- 2026 so far0
Abstract
Introduction Patients increasingly use large language models (LLMs) for health information, but their suitability for high-risk reproductive medicine remains uncertain. We compared six LLMs on patient-facing questions about asthma during pregnancy across clinician-rated overall clinical response quality, information quality, understandability, actionability, readability, and safety. Methods In this cross-sectional comparative study, a two-stage modified Delphi-informed process reduced an initial 40-question pool to 30 candidates and 19 retained questions. Six LLMs answered each question once under a standardized structured prompt, yielding 114 responses and 342 rater-level records. Three clinical evaluators assessed clinician-rated overall clinical response quality, DISCERN, and PEMAT understandability and actionability; four readability indices were calculated. Models were compared using matched question-level nonparametric tests with multiplicity correction and effect sizes. In a revision-stage safety analysis, two clinicians independently rated all 114 responses using a structured safety taxonomy; responses with rating disagreements or prespecified severity triggers underwent blinded adjudication by a third senior clinician. Results Clinician-rated overall clinical response quality differed across models [Friedman χ 2 (5, N = 19) = 72.31, P < 0.001; Kendall's W = 0.76]. GPT-5.5 and Claude Sonnet 5 had the highest observed median scores, without a significant difference after correction. DISCERN, PEMAT, and readability also differed across models, but no model led consistently across outcomes. None achieved a median Flesch-Kincaid grade level at or below grade 8. Inter-rater agreement for the presence of any safety concern was 87.7% (Cohen's kappa=0.70). After adjudication by a third senior clinician, 37/114 responses (32.5%) had at least one safety concern, including 4/114 (3.5%) rated as high-risk unsafe advice. Conclusions Under structured prompting, LLM performance was promising but uneven across selected asthma-in-pregnancy questions. High clinician-rated overall clinical response quality did not ensure superior information quality, usability, readability, or safety. The results do not establish safety for unsupervised patient use. LLM-generated patient information should undergo guideline-based clinical verification, explicit safety review, and readability editing before deployment.
What the data shows
- No database records a citation yet, as is usual for a paper this new.
Graph
This paper in the middle, its authors and their most-cited papers below, the papers it cites to the left and the papers citing it to the right. The timeline lays every paper out by year.
Select a node for its details. Drag to pan, scroll to zoom, drag a node to move it.
Start with
Cited by 0 papers
Who cites this paper, from open citation data, with Crossref's record of each; the 100 newest when there are more.
References (24)
What it cites, from open citation data, with Crossref's record of each.
Sign in to see 4 more references
A free account opens the rest of the citing papers and references on every paper’s page.