Surgical Innovation · 6 Nov 2026
Deep-Learning Enhanced Augmented Reality for Robotic Radical Prostatectomy on Consumer Grade Hardware
Vincent Benardh-index 0, Olivier Windisch, Mina Bjelogrlic, Hugues Turbé, Christian Lovis, Michele Diana, Massimo Valerio
Citations by database
Each database indexes a different set of journals, preprints and books, so they are shown side by side rather than blended.
- OpenCitations0
- Crossref0
- Europe PMC0
Citations per year
The current year is still counting.
- 2026 so far0
Abstract
Background Augmented Reality (AR) during Robotic-Assisted Radical Prostatectomy (RARP) provides surgical guidance by overlaying patient-specific 3D Virtual Models (VM) onto the intraoperative view. A main drawback is instrument occlusion, where VM overlaps the surgical tools, disturbing the augmented view. Deep learning (DL)-based desocclusion addresses this issue, but current implementation rely on expensive, specialized hardware. Methods We developed an AR desocclusion pipeline running on consumer-grade hardware (∼1150 US Dollars). For model selection, we trained and benchmarked 21 encoder architectures paired with a U-Net decoder. The selected model was integrated into a macOS application using CoreML inference on the Apple Neural Engine. Memory usage, thermal state and framerate were monitored on a continuous 81-minute console session. Photon-to-photon latency (from surgical field to the surgeon eye) was measured on the DaVinci® Xi using a microcontroller-phototransistor setup. Results FastViT-S12/Unet offered the highest accuracy among models meeting our inference-time budget (∼2 ms inference), achieving a Dice score of 95.1% (95% CI, 94.3-95.7) internally, 91.6% (95% CI, 90.2-92.9) and 90.2% (95% CI, 84.7-94.4) on externals tests, with a mean inference time of 1.89 ± 0.36 ms. The pipeline maintained 57.98 frames per second using 1.47 GB of unified memory with a nominal thermal state. On the surgical console, photon-to-photon median latency increased from 60 ms (Q1-Q3: 57.5-62.5) to 120.5 ms (Q1-Q3: 111-131) with the AR pipeline. Conclusions Real-time AR-RARP with DL-based desocclusion is achievable on low-cost hardware with an added latency remaining within ranges reported as technically acceptable for intraoperative use.
What the data shows
- No database records a citation yet, as is usual for a paper this new.
- Vincent Benard has the highest h-index among the authors (0).
Graph
This paper in the middle, its authors and their most-cited papers below, the papers it cites to the left and the papers citing it to the right. The timeline lays every paper out by year.
Select a node for its details. Drag to pan, scroll to zoom, drag a node to move it.
Start with
Cited by 0 papers
Who cites this paper, from open citation data, with Crossref's record of each; the 100 newest when there are more.
References (32)
What it cites, from open citation data, with Crossref's record of each.
Sign in to see 12 more references
A free account opens the rest of the citing papers and references on every paper’s page.