Journal of Medical Internet Research · 11 Nov 2026
Diagnostic Accuracy and Fine-Grained Image-Interpretation Performance of Multimodal Large Language Models on Bilingual Radiology Questions: Cross-Sectional Benchmark Study
Yijie Liuh-index 0, Xin Zhang, Jia Xu, Manxin Huang, Yinting Xiong, Wenqian Wuh-index 11
Citations by database
Each database indexes a different set of journals, preprints and books, so they are shown side by side rather than blended.
- OpenCitations0
- Crossref0
Citations per year
The current year is still counting.
- 2026 so far0
Abstract
Abstract Background Multimodal large language models (LLMs) are rapidly expanding in radiology and digital health; however, their diagnostic performance across diverse clinical scenarios and fine-grained visual reasoning tasks requires systematic evaluation. Objective The study aims to evaluate the diagnostic accuracy and fine-grained visual reasoning performance of 5 multimodal LLMs with bilingual medical imaging questions. Methods This cross-sectional benchmark study evaluated bilingual medical imaging questions using zero-shot prompting on 5 LLMs: GPT-5.2, Gemini 3 Pro, Doubao, Yuanbao, and Llama 4 Maverick. Subgroup analyses by language (English/Chinese), question type (image/text), imaging modality (computed tomography, radiography, magnetic resonance imaging, ultrasound, and nuclear medicine), and anatomical system (neurologic, respiratory, cardiovascular, digestive, genitourinary, and musculoskeletal) evaluated intermodel accuracy and intramodel consistency. Image-based visual reasoning performance was assessed across modality, anatomy, lesion detection, and localization. Results A total of 298 questions (198 image-based, 100 text-based) were evaluated. Gemini 3 Pro achieved the highest overall accuracy (n=268, 89.9%, 95% CI 86.0%‐92.9%), significantly outperforming Doubao (n=235, 78.9%; P <.001), GPT-5.2 (n=233, 78.2%; P <.001), Yuanbao (n=216, 72.5%; P <.001), and Llama 4 Maverick (n=216, 72.5%; P <.001). All models performed better on text-based than image-based questions (Yuanbao, P <.001; Doubao, P =.02; GPT-5.2, P <.001; Gemini 3 Pro, P =.002; Llama 4 Maverick, P <.001). GPT-5.2 ( P <.001), Llama 4 Maverick ( P =.03), and Yuanbao ( P =.03) favored English questions, while intramodel performance remained stable across modalities (Yuanbao, P =.46; Doubao, P =.21; GPT-5.2, P =.26; Gemini 3 Pro, P =.96; Llama 4 Maverick, P =.45) and anatomical systems (Yuanbao, P =.82; Doubao, P =.52; GPT-5.2, P =.35; Gemini 3 Pro, P =.13; Llama 4 Maverick, P =.97). Despite near-perfect macroscopic visual recognition (modality and anatomy: 99.0%‐100.0%), accuracy dropped in complex tasks. Gemini 3 Pro outperformed Yuanbao ( P <.001), GPT-5.2 ( P =.002), and Llama 4 Maverick ( P <.001) in lesion detection (175/198, 88.4%) and outperformed Yuanbao ( P <.001), Doubao ( P =.004), and Llama 4 Maverick ( P =.004) in lesion localization (189/198, 95.5%). Conclusions Gemini 3 Pro demonstrated optimal, comprehensive performance and cross-lingual stability. However, current multimodal LLMs face a technical bottleneck in complex visual interpretation—particularly in fine-grained lesion detection and localization—emphasizing the need to prioritize local feature perception in future model development.
What the data shows
- No database records a citation yet, as is usual for a paper this new.
- Wenqian Wu has the highest h-index among the authors (11).
Graph
This paper in the middle, its authors and their most-cited papers below, the papers it cites to the left and the papers citing it to the right. The timeline lays every paper out by year.
Select a node for its details. Drag to pan, scroll to zoom, drag a node to move it.
Start with
Cited by 0 papers
Who cites this paper, from open citation data, with Crossref's record of each; the 100 newest when there are more.
References (26)
What it cites, from open citation data, with Crossref's record of each.
Sign in to see 6 more references
A free account opens the rest of the citing papers and references on every paper’s page.