Computer Life · 13 Nov 2026
Execution-Guided Verification for Reliable Tool-Using Large Language Model Agents
Oliver Hargreaves, Sienna Patel, Callum Price, Emily Foster
Citations by database
Each database indexes a different set of journals, preprints and books, so they are shown side by side rather than blended.
- OpenCitations0
- Crossref0
Citations per year
The current year is still counting.
- 2026 so far0
Abstract
Large language model (LLM) agents increasingly operate on real systems through tool calls, yet their behavior is stochastic: the same request may be resolved correctly in one run and incorrectly in the next, and erroneous write operations are costly to undo. This paper presents execution-guided verification (EGV), a training-free and model-agnostic procedure that decides which agent behavior to commit by executing candidate trajectories instead of judging their text. EGV replays the tool calls of k independently sampled trajectories against isolated copies of the backend state, groups candidates that reach an identical final world state, and commits a representative of the best-supported state, while failed tool calls serve as a secondary signal for tie-breaking and representative selection. The support share of the committed state acts as a verification confidence for selective execution with escalation to human operators. Experiments on 1,980 real trajectories of GPT-4o and Claude-3.5-Sonnet in the retail and airline domains of τ-bench, replayed in a sandbox that reproduces 99.96% of the recorded tool observations, show that EGV improves task success over single-run execution in every setting, e.g., from 61.0% to 68.7% for Claude-3.5-Sonnet in retail, with a pooled gain of 6.1 points (95% CI [3.8, 8.3]), and that it consistently outperforms voting over emitted action sequences. Execution agreement predicts trajectory success with an AUROC of 79.4–82.8, and committing only unanimously supported states yields 81.0–100% success on 16–40% of the requests. These results establish world-state equivalence as an effective and inexpensive verification signal for stateful tool-using agents.
What the data shows
- No database records a citation yet, as is usual for a paper this new.
Graph
This paper in the middle, its authors and their most-cited papers below, the papers it cites to the left and the papers citing it to the right. The timeline lays every paper out by year.
Select a node for its details. Drag to pan, scroll to zoom, drag a node to move it.
Start with
Cited by 0 papers
Who cites this paper, from open citation data, with Crossref's record of each; the 100 newest when there are more.
References (61)
What it cites, from open citation data, with Crossref's record of each.
Sign in to see 41 more references
A free account opens the rest of the citing papers and references on every paper’s page.