OpenResearchAlpha

Journal of Clinical Medicine · 13 Nov 2026

Comparative Expert-Rated Response Quality of Large Language Models in Aneurysmal Subarachnoid Hemorrhage: A Multidimensional Evaluation

Yasin Taşkınh-index 1, Ahmet Tuğrul Şahinh-index 3

0
Citations
no database records one yet
0
Citations in 2026
so far
3
Leading author's h-index
Ahmet Tuğrul Şahin, in QAI's scholar index
23
References
works it cites, from open citation data

Citations by database

Each database indexes a different set of journals, preprints and books, so they are shown side by side rather than blended.

  • OpenCitations0
  • Crossref0

Citations per year

The current year is still counting.

  • 2026 so far0

Abstract

Background/Objectives: Large language models (LLMs) are increasingly used to obtain medical information, but the quality of their responses to complex, time-sensitive neurovascular questions requires disease-specific evaluation. We compared the expert-rated response quality of five publicly accessible LLM systems using standardized questions on aneurysmal subarachnoid hemorrhage (aSAH). Methods: Thirty standardized Turkish-language questions covering aneurysm treatment, vasospasm and related complications, and neurocritical care and follow-up (10 questions per domain) were submitted to ChatGPT, Gemini, Claude, DeepSeek, and Microsoft Copilot through public web interfaces in July 2026. Each question was submitted once per system in a separate chat session, generating 150 responses. Five senior neurosurgeons independently rated each response on a 1–10 scale for coverage, relevance, comprehensibility, clinical applicability, and overall quality. Inter-rater reliability was assessed using intraclass correlation coefficients (ICCs). Repeated-measures linear mixed models, Bonferroni-adjusted pairwise comparisons, and Friedman sensitivity analyses were performed. Results: The evaluation comprised 3750 expert score observations without missing data. Inter-rater reliability was good to excellent (ICC 0.835–0.911). AI system was significantly associated with all five evaluation dimensions (all p < 0.001). Adjusted overall scores were highest for ChatGPT (7.68), followed by Gemini (6.97), Claude (6.69), DeepSeek (5.92), and Copilot (4.75). Neither the clinical-topic main effect nor the model-by-topic interaction was statistically significant (all p > 0.05). Friedman analyses also identified between-model differences across all dimensions (all p < 0.001). Conclusions: Publicly accessible LLM systems differed significantly in expert-rated response quality when answering standardized Turkish-language aSAH questions. These differences were observed within the study’s specific question set and evaluation protocol and should not be interpreted as evidence of differences in real-world clinical outcomes. The absence of statistically significant model-by-topic interactions does not establish equivalent performance across clinical domains. Model-specific validation, assessment of clinically important errors, and continued expert oversight are needed before considering LLM-generated information for clinical decision support.

What the data shows

  • No database records a citation yet, as is usual for a paper this new.
  • Ahmet Tuğrul Şahin has the highest h-index among the authors (3).

Graph

This paper in the middle, its authors and their most-cited papers below, the papers it cites to the left and the papers citing it to the right. The timeline lays every paper out by year.

Citing paper shares an authorAuthor without a profileSize shows citations, or h-index for authors

Select a node for its details. Drag to pan, scroll to zoom, drag a node to move it.

Start with

Authors (2)

Each author's standing in QAI's scholar index: the works their ORCID iD is attached to, and for anyone whose ORCID record names a lab OpenResearch follows, their arXiv preprints matched by name and a shared co-author. Names are as printed on the paper.

ProfileMost-cited paper

Cited by 0 papers

Who cites this paper, from open citation data, with Crossref's record of each; the 100 newest when there are more.

0 shown
No citing papers for this filter.

References (23)

What it cites, from open citation data, with Crossref's record of each.

Sign in to see 3 more references

A free account opens the rest of the citing papers and references on every paper’s page.

Comparative Expert-Rated Response Quality of Large Language Models in Aneurysmal Subarachnoid Hemorrhage: A Multidimensional Evaluation - QAI OpenResearch