Journal of Clinical Medicine · 13 Nov 2026
Comparative Expert-Rated Response Quality of Large Language Models in Aneurysmal Subarachnoid Hemorrhage: A Multidimensional Evaluation
Yasin Taşkınh-index 1, Ahmet Tuğrul Şahinh-index 3
Citations by database
Each database indexes a different set of journals, preprints and books, so they are shown side by side rather than blended.
- OpenCitations0
- Crossref0
Citations per year
The current year is still counting.
- 2026 so far0
Abstract
Background/Objectives: Large language models (LLMs) are increasingly used to obtain medical information, but the quality of their responses to complex, time-sensitive neurovascular questions requires disease-specific evaluation. We compared the expert-rated response quality of five publicly accessible LLM systems using standardized questions on aneurysmal subarachnoid hemorrhage (aSAH). Methods: Thirty standardized Turkish-language questions covering aneurysm treatment, vasospasm and related complications, and neurocritical care and follow-up (10 questions per domain) were submitted to ChatGPT, Gemini, Claude, DeepSeek, and Microsoft Copilot through public web interfaces in July 2026. Each question was submitted once per system in a separate chat session, generating 150 responses. Five senior neurosurgeons independently rated each response on a 1–10 scale for coverage, relevance, comprehensibility, clinical applicability, and overall quality. Inter-rater reliability was assessed using intraclass correlation coefficients (ICCs). Repeated-measures linear mixed models, Bonferroni-adjusted pairwise comparisons, and Friedman sensitivity analyses were performed. Results: The evaluation comprised 3750 expert score observations without missing data. Inter-rater reliability was good to excellent (ICC 0.835–0.911). AI system was significantly associated with all five evaluation dimensions (all p < 0.001). Adjusted overall scores were highest for ChatGPT (7.68), followed by Gemini (6.97), Claude (6.69), DeepSeek (5.92), and Copilot (4.75). Neither the clinical-topic main effect nor the model-by-topic interaction was statistically significant (all p > 0.05). Friedman analyses also identified between-model differences across all dimensions (all p < 0.001). Conclusions: Publicly accessible LLM systems differed significantly in expert-rated response quality when answering standardized Turkish-language aSAH questions. These differences were observed within the study’s specific question set and evaluation protocol and should not be interpreted as evidence of differences in real-world clinical outcomes. The absence of statistically significant model-by-topic interactions does not establish equivalent performance across clinical domains. Model-specific validation, assessment of clinically important errors, and continued expert oversight are needed before considering LLM-generated information for clinical decision support.
What the data shows
- No database records a citation yet, as is usual for a paper this new.
- Ahmet Tuğrul Şahin has the highest h-index among the authors (3).
Graph
This paper in the middle, its authors and their most-cited papers below, the papers it cites to the left and the papers citing it to the right. The timeline lays every paper out by year.
Select a node for its details. Drag to pan, scroll to zoom, drag a node to move it.
Start with
Cited by 0 papers
Who cites this paper, from open citation data, with Crossref's record of each; the 100 newest when there are more.
References (23)
What it cites, from open citation data, with Crossref's record of each.
Sign in to see 3 more references
A free account opens the rest of the citing papers and references on every paper’s page.