OpenResearchAlpha

arXiv · 11 Mar 2026

IH-Challenge: A Training Dataset to Improve Instruction Hierarchy on Frontier LLMs

Chuan Guo, Juan Felipe Ceron Uribe, Sicheng Zhu, Christopher A. Choquette-Choo, Steph Lin, Nikhil Kandpal, Milad Nasr, Rai, Sam Toyer, Miles Wang, Yaodong Yu, Alex Beutelh-index 5, Kai Xiaoh-index 3

21
Citations
OpenAlex's count; others 0
0
Citations in 2026
so far
5
Leading author's h-index
Alex Beutel, in QAI's scholar index
0
References
works it cites, from open citation data

Citations by database

Each database indexes a different set of journals, preprints and books, so they are shown side by side rather than blended.

  • OpenAlex21
  • OpenCitations0

Citations per year

The current year is still counting.

  • 2026 so far0

Abstract

Instruction hierarchy (IH) defines how LLMs prioritize system, developer, user, and tool instructions under conflict, providing a concrete, trust-ordered policy for resolving instruction conflicts. IH is key to defending against jailbreaks, system prompt extractions, and agentic prompt injections. However, robust IH behavior is difficult to train: IH failures can be confounded with instruction-following failures, conflicts can be nuanced, and models can learn shortcuts such as overrefusing. We introduce IH-Challenge, a reinforcement learning training dataset, to address these difficulties. Fine-tuning GPT-5-Mini on IH-Challenge with online adversarial example generation improves IH robustness by +10.0% on average across 16 in-distribution, out-of-distribution, and human red-teaming benchmarks (84.1% to 94.1%), reduces unsafe behavior from 6.6% to 0.7% while improving helpfulness on general safety evaluations, and saturates an internal static agentic prompt injection evaluation, with minimal capability regression. We release the IH-Challenge dataset (https://huggingface.co/datasets/openai/ih-challenge) to support future research on robust instruction hierarchy.

Artificial intelligenceComputation and languageCryptography and securityMachine learning

What the data shows

  • OpenAlex counts 21 citations, the most of the databases. The others count 0.
  • Alex Beutel has the highest h-index among the authors (5).

Graph

This paper in the middle, its authors and their most-cited papers below, the papers it cites to the left and the papers citing it to the right. The timeline lays every paper out by year.

Citing paper shares an authorAuthor without a profileSize shows citations, or h-index for authors

Select a node for its details. Drag to pan, scroll to zoom, drag a node to move it.

Start with

Authors (13)

Each author's standing in QAI's scholar index: the works their ORCID iD is attached to, and for anyone whose ORCID record names a lab OpenResearch follows, their arXiv preprints matched by name and a shared co-author. Names are as printed on the paper.

ProfileMost-cited paper

Cited by 0 papers

Who cites this paper, from open citation data, with Crossref's record of each; the 100 newest when there are more.

0 shown
No citing papers for this filter.

References (0)

What it cites, from open citation data, with Crossref's record of each.

No references in the open citation data yet.