Journal of Medical Internet Research · 13 Nov 2026
Hallucinated Evidence and Real-World Risk—Physicians’ Intent to Act on AI-Generated Abstracts in Emergency Medicine and Trauma Surgery: Cross-Sectional, Vignette-Based Diagnostic Accuracy Study
Hassan Tarek Hakamh-index 5, Nicolas Vuillerme, Felix Muehlensiepen, Nikolai Ramadanov, Jonathan Lettner, Marko Ostojic, Roland Becker, Robert Prillh-index 16
Citations by database
Each database indexes a different set of journals, preprints and books, so they are shown side by side rather than blended.
- OpenCitations0
- Crossref0
- Europe PMC0
Citations per year
The current year is still counting.
- 2026 so far0
Abstract
Background Large language models (LLMs) are increasingly used in academic writing and can generate linguistically polished but factually incorrect scientific content. Such misinformation may distort evidence appraisal and, if translated into clinical practice, compromise patient care. Objective This study aimed to estimate physicians’ likelihood of misperceiving false information embedded in LLM-generated abstracts as true and reporting an intent to use that information in future practice. We also explored which features were associated with perceived credibility and intended clinical action. Methods We conducted a cross-sectional, vignette-based survey of physicians working in emergency medicine and trauma surgery across 4 hospitals in Germany. Each participant assessed 4 abstracts generated using GPT-5 Thinking mode, of which 2 contained intentionally false claims supported by plausible explanations and 2 contained accurate claims. The primary outcome was the proportion of physicians reporting an intent to use or act on recommendations contained in abstracts with false claims. Secondary outcomes included perceived accuracy, rejection of accurate claims, and ratings of precision, structure, and grammar. Results Twenty physicians provided 80 ratings of 4 abstracts. Overall, physicians reported that they would act on 68% (54/80) of abstracts. Using abstract truth status as the reference, the decision to act had a sensitivity of 78% (31/40), specificity of 43% (17/40), positive predictive value of 57% (31/54), and negative predictive value of 65% (17/26). The positive likelihood ratio was 1.35 (95% CI 0.98-1.85), the negative likelihood ratio was 0.53 (95% CI 0.27-1.04), and the diagnostic odds ratio was 2.55 (95% CI 0.96-6.73). True and false abstracts received similar ratings for perceived accuracy (mean 3.15, SD 0.89 vs mean 2.88, SD 1.04; P=.21), precision (mean 3.15, SD 0.89 vs mean 2.93, SD 1.00; P=.29), structure (mean 3.23, SD 0.77 vs mean 3.20, SD 0.88; P=.89), and grammar (mean 3.40, SD 0.59 vs mean 3.43, SD 0.68; P=.86). Abstracts that physicians intended to act on were rated substantially higher for perceived accuracy, precision, and structure than those they would not act on (all P<.001), whereas grammar ratings did not differ (P=.80). Conclusions Physicians frequently reported an intent to act on LLM-generated abstracts containing false claims, and true and false abstracts were not reliably distinguished on perceived quality measures. Perceived accuracy, precision, and structure were strongly associated with intended action, whereas grammar was not. These findings highlight the risk that persuasive scientific presentation may increase the credibility and potential clinical influence of fabricated or inaccurate AI-generated evidence.
What the data shows
- No database records a citation yet, as is usual for a paper this new.
- Robert Prill has the highest h-index among the authors (16).
Graph
This paper in the middle, its authors and their most-cited papers below, the papers it cites to the left and the papers citing it to the right. The timeline lays every paper out by year.
Select a node for its details. Drag to pan, scroll to zoom, drag a node to move it.
Start with
Cited by 0 papers
Who cites this paper, from open citation data, with Crossref's record of each; the 100 newest when there are more.
References (100)
What it cites, from open citation data, with Crossref's record of each.
Sign in to see 80 more references
A free account opens the rest of the citing papers and references on every paper’s page.