OpenResearchAlpha

Journal of the American College of Surgeons · 18 Nov 2026

Role of Surgeon-Model Interaction in the Real-World Performance of Surgical Risk Prediction Tools

Sofia Volpi, Tyler Loftus, Jayson Marwaha, Margaret Berrigan, Zeynep Bilgili, Anupamaa Seshadri, Daniel Bent, Stephen Odom, Charles Parsons, Alok Gupta, Charles Cook, Azra Bihorac, Gabriel Brat

0
Citations
no database records one yet
0
Citations in 2026
so far
–
Leading author's h-index
no author has a profile yet
0
References
works it cites, from open citation data

Citations by database

Each database indexes a different set of journals, preprints and books, so they are shown side by side rather than blended.

  • OpenCitations0
  • Crossref0

Citations per year

The current year is still counting.

  • 2026 so far0

Abstract

Background: Surgical risk models are typically evaluated independently of how surgeons interpret their predictions after deployment. We evaluated surgeon-model disagreement across distinct clinical environments and whether surgeon responses added prognostic information. Study Design: In a multi-institutional study, surgeons reviewed 30-day complication risks generated by locally deployed NSQIP-SRC, POTTER, or MySurgeryRisk models and assessed risk relative to model estimates. We evaluated disagreement, observed outcomes, and model discrimination overall, among discordant assessments, and across model-specific risk quartiles. Logistic regression models incorporating model predictions and surgeon responses underwent 5-fold cross-validation, with discrimination compared using DeLong’s test. Results: Thirty-one surgeons evaluated 1,702 predictions across 2 academic medical centers. Surgeons disagreed with 53% of predictions, assessing risk as higher in 37% and lower in 16%. Lower-than-model assessments were followed by no complication in 84% of cases, whereas higher-than-model assessments were followed by a complication in 23%. Model AUCs were 0.66, 0.82, and 0.83 for NSQIP-SRC, POTTER, and MySurgeryRisk, respectively. Among discordant assessments, AUCs were 0.61, 0.79, and 0.83, respectively. Incorporating surgeon responses increased AUCs to 0.68, 0.83, and 0.86, respectively; none of these differences were statistically significant. Effects varied across risk quartiles. Conclusions: Surgeons frequently disagreed with deployed surgical risk models, but disagreement was not uniformly beneficial or consistently associated with poorer model discrimination. Evaluating how clinicians interpret and modify model predictions may be important for understanding real-world performance beyond conventional model metrics.

What the data shows

  • No database records a citation yet, as is usual for a paper this new.

Graph

This paper in the middle, its authors and their most-cited papers below, the papers it cites to the left and the papers citing it to the right. The timeline lays every paper out by year.

Citing paper shares an authorAuthor without a profileSize shows citations, or h-index for authors

Select a node for its details. Drag to pan, scroll to zoom, drag a node to move it.

Start with

Authors (13)

Each author's standing in QAI's scholar index: the works their ORCID iD is attached to, and for anyone whose ORCID record names a lab OpenResearch follows, their arXiv preprints matched by name and a shared co-author. Names are as printed on the paper.

ProfileMost-cited paper

Cited by 0 papers

Who cites this paper, from open citation data, with Crossref's record of each; the 100 newest when there are more.

0 shown
No citing papers for this filter.

References (0)

What it cites, from open citation data, with Crossref's record of each.

No references in the open citation data yet.