OpenResearchAlpha

Information · 20 Nov 2026

A Controlled Behavioral Benchmark for Diagnosing Atomic Accessibility and Compositional Utilization in Large Language Models

Eryang Huanh-index 2

0
Citations
no database records one yet
0
Citations in 2026
so far
2
Leading author's h-index
Eryang Huan, in QAI's scholar index
23
References
works it cites, from open citation data

Citations by database

Each database indexes a different set of journals, preprints and books, so they are shown side by side rather than blended.

  • OpenCitations0
  • Crossref0

Citations per year

The current year is still counting.

  • 2026 so far0

Abstract

Large language models are increasingly deployed as reasoning and decision support components in intelligent systems, yet aggregate benchmark scores provide limited evidence about whether they can reliably access primitive operations, combine them compositionally, adapt to reformulated contexts, and preserve successful behavior under perturbation. We introduce PPC-Bench, a controlled behavioral benchmark comprising 5000 deterministic samples across mathematical computation, Python code execution, and rule-based reasoning. Nine open-source models are evaluated along five dimensions: Direct Accessibility, Compositional Utilization, Prompt Elicitation, Contextual Adaptation, and Robustness Consistency. To distinguish inaccessible primitive capability from failed composition, we further construct a separate paired set of 1800 samples organized into 240 matched task families. Each family aligns atomic probes with composition, prompted, contextual, and perturbed variants of the same task. Qwen3.5-27B correctly answers 839 of 840 atomic probes but successfully composes only 74 of 239 atom-accessible families. In contrast, Llama-3.1-8B-Instruct correctly composes 132 of 164 atom-accessible families despite lower atomic accuracy. Gemma-3-1B-IT exhibits limitations in both atomic accessibility and compositional utilization. Step-by-step prompting recovers approximately half of the accessible-but-uncomposed families for Qwen3.5-27B and Llama-3.1-8B-Instruct, while successful composition is preserved under perturbation in more than 91% of eligible families for both models. These results provide condition-specific behavioral profiles that distinguish initial composition success, recovery of composition failures, and correctness preservation within successful subsets. PPC-Bench combines broad model-level evaluation with matched-family conditional analysis, providing a basis for investigating model and prompt choices without inferring internal mechanisms from task accuracy.

What the data shows

  • No database records a citation yet, as is usual for a paper this new.
  • Eryang Huan has the highest h-index among the authors (2).

Graph

This paper in the middle, its authors and their most-cited papers below, the papers it cites to the left and the papers citing it to the right. The timeline lays every paper out by year.

Citing paper shares an authorAuthor without a profileSize shows citations, or h-index for authors

Select a node for its details. Drag to pan, scroll to zoom, drag a node to move it.

Start with

Authors (1)

Each author's standing in QAI's scholar index: the works their ORCID iD is attached to, and for anyone whose ORCID record names a lab OpenResearch follows, their arXiv preprints matched by name and a shared co-author. Names are as printed on the paper.

ProfileMost-cited paper

Cited by 0 papers

Who cites this paper, from open citation data, with Crossref's record of each; the 100 newest when there are more.

0 shown
No citing papers for this filter.

References (23)

What it cites, from open citation data, with Crossref's record of each.

Sign in to see 3 more references

A free account opens the rest of the citing papers and references on every paper’s page.

A Controlled Behavioral Benchmark for Diagnosing Atomic Accessibility and Compositional Utilization in Large Language Models - QAI OpenResearch