OpenResearchAlpha

Journal of Medical Internet Research · 11 Nov 2026

Diagnostic Accuracy and Fine-Grained Image-Interpretation Performance of Multimodal Large Language Models on Bilingual Radiology Questions: Cross-Sectional Benchmark Study

Yijie Liuh-index 0, Xin Zhang, Jia Xu, Manxin Huang, Yinting Xiong, Wenqian Wuh-index 11

0
Citations
no database records one yet
0
Citations in 2026
so far
11
Leading author's h-index
Wenqian Wu, in QAI's scholar index
26
References
works it cites, from open citation data

Citations by database

Each database indexes a different set of journals, preprints and books, so they are shown side by side rather than blended.

  • OpenCitations0
  • Crossref0

Citations per year

The current year is still counting.

  • 2026 so far0

Abstract

Abstract Background Multimodal large language models (LLMs) are rapidly expanding in radiology and digital health; however, their diagnostic performance across diverse clinical scenarios and fine-grained visual reasoning tasks requires systematic evaluation. Objective The study aims to evaluate the diagnostic accuracy and fine-grained visual reasoning performance of 5 multimodal LLMs with bilingual medical imaging questions. Methods This cross-sectional benchmark study evaluated bilingual medical imaging questions using zero-shot prompting on 5 LLMs: GPT-5.2, Gemini 3 Pro, Doubao, Yuanbao, and Llama 4 Maverick. Subgroup analyses by language (English/Chinese), question type (image/text), imaging modality (computed tomography, radiography, magnetic resonance imaging, ultrasound, and nuclear medicine), and anatomical system (neurologic, respiratory, cardiovascular, digestive, genitourinary, and musculoskeletal) evaluated intermodel accuracy and intramodel consistency. Image-based visual reasoning performance was assessed across modality, anatomy, lesion detection, and localization. Results A total of 298 questions (198 image-based, 100 text-based) were evaluated. Gemini 3 Pro achieved the highest overall accuracy (n=268, 89.9%, 95% CI 86.0%‐92.9%), significantly outperforming Doubao (n=235, 78.9%; P <.001), GPT-5.2 (n=233, 78.2%; P <.001), Yuanbao (n=216, 72.5%; P <.001), and Llama 4 Maverick (n=216, 72.5%; P <.001). All models performed better on text-based than image-based questions (Yuanbao, P <.001; Doubao, P =.02; GPT-5.2, P <.001; Gemini 3 Pro, P =.002; Llama 4 Maverick, P <.001). GPT-5.2 ( P <.001), Llama 4 Maverick ( P =.03), and Yuanbao ( P =.03) favored English questions, while intramodel performance remained stable across modalities (Yuanbao, P =.46; Doubao, P =.21; GPT-5.2, P =.26; Gemini 3 Pro, P =.96; Llama 4 Maverick, P =.45) and anatomical systems (Yuanbao, P =.82; Doubao, P =.52; GPT-5.2, P =.35; Gemini 3 Pro, P =.13; Llama 4 Maverick, P =.97). Despite near-perfect macroscopic visual recognition (modality and anatomy: 99.0%‐100.0%), accuracy dropped in complex tasks. Gemini 3 Pro outperformed Yuanbao ( P <.001), GPT-5.2 ( P =.002), and Llama 4 Maverick ( P <.001) in lesion detection (175/198, 88.4%) and outperformed Yuanbao ( P <.001), Doubao ( P =.004), and Llama 4 Maverick ( P =.004) in lesion localization (189/198, 95.5%). Conclusions Gemini 3 Pro demonstrated optimal, comprehensive performance and cross-lingual stability. However, current multimodal LLMs face a technical bottleneck in complex visual interpretation—particularly in fine-grained lesion detection and localization—emphasizing the need to prioritize local feature perception in future model development.

What the data shows

  • No database records a citation yet, as is usual for a paper this new.
  • Wenqian Wu has the highest h-index among the authors (11).

Graph

This paper in the middle, its authors and their most-cited papers below, the papers it cites to the left and the papers citing it to the right. The timeline lays every paper out by year.

Citing paper shares an authorAuthor without a profileSize shows citations, or h-index for authors

Select a node for its details. Drag to pan, scroll to zoom, drag a node to move it.

Start with

Authors (6)

Each author's standing in QAI's scholar index: the works their ORCID iD is attached to, and for anyone whose ORCID record names a lab OpenResearch follows, their arXiv preprints matched by name and a shared co-author. Names are as printed on the paper.

ProfileMost-cited paper

Cited by 0 papers

Who cites this paper, from open citation data, with Crossref's record of each; the 100 newest when there are more.

0 shown
No citing papers for this filter.

References (26)

What it cites, from open citation data, with Crossref's record of each.

Sign in to see 6 more references

A free account opens the rest of the citing papers and references on every paper’s page.