arXiv AI By Adarsh Agrawal (Independent Researcher)

When Correctness Feedback Erases Evidence: Phase-Separated Completeness Auditing in Verus

Read the original on arXiv AI →

The paper shows that providing correctness feedback in the Verus/Rust verification workflow actually removes the evidence needed for completeness auditing, leading to a significant drop in certified witness yield compared to no‑feedback approaches. Experiments on 12 tasks across four model families demonstrate that independent discovery without feedback more than doubles the yield (59.0% vs. 27.1%) and that a phase‑separated execution can recover this yield without reusing responses. Preserving these witnesses also improves specification repair, with Phase‑Revised contracts achieving 81.3% implementation success versus 62.5% for IS‑Revised, while the effect does not appear in Dafny or Lean evaluations, indicating a specific limitation of the Verus/Rust workflow.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Oct 2

ContractRL: Shielded Group-Relative Policy Optimization for Auditable Tool-Call Repair

ContractRL introduces a contract-constrained sequential repair protocol for structured tool calls, modeling verifier-guided JSON repair as a bounded decision process. The policy observes candidate data, verifier feedback, JSON pointers, repair history, and budget, using a contract-derived action mask to filter invalid operations before a deterministic validator applies changes. Compared to Patch‑SFT and full regeneration, ContractRL achieves higher semantic success (0.9362 vs. 0.9076 and 0.9148) while generating fewer tokens (34.4 vs. 44.9 and 137.2), and policy optimization further improves success rates.

By Miaobo Hu, Shuhao Hu, Xiaobo Guo, Xin Wang, Bokun Wang, Yina Sa, Daren Zha, Jun Xiao
arXiv AI
Sep 15

Externalizing Requirement-to-Repair Artifacts as Observable Traces for LLM-Based Program Repair

The paper introduces THEMIS, a stage-aware repair workflow that externalizes the requirement-to-repair process by generating semantic interpretations, a runtime requirement-code graph, graph-derived developer guidance, retained repair rationale and patches, and post-edit audit records. A retrospective audit of 300 SWE-bench Lite cases shows that these artifacts enable cross-stage inspection, with a complete developer rationale available for 288 cases and 214 cases retaining a full audited field set. The retained records also allow systematic measurement of cross-stage correspondence, revealing high recurrence of target symbols across rationales and patches, and a preliminary improvement in resolving cases compared to a direct same-input condition.

By Zewen Tao, Shin-nosuke Ishikawa
arXiv Computation and Language
Sep 24

Exact Feedback Is Not Control: Evaluating Text-based Closed-Loop Revision in LLMs

The paper introduces a fixed‑budget revision protocol that uses deterministic verifiers to expose all remaining violations across exact‑length, lexical, and compositional constraints, thereby isolating model‑side revision behavior. Experiments on 19 open‑ and closed‑source LLMs show wide variability in controller‑level success, with some models achieving up to 99.8% success while others remain below 20%. Controlled studies reveal that post‑training and scale affect model responses to exact feedback, but do not consistently improve exact correction, and that recurrence of earlier outputs is linked to lower recoverability.

By Haitong Jiang, Chunlin Liu, Yile Wang, Yuhong Feng
arXiv AI
Sep 2

REVISE: Validity-Guided Recovery for Online Revisions in Agent Workflows

The paper introduces “Revise”, a runtime system that performs validity-guided, fine-grained recovery for online revisions in structured agent workflows. When a revision arrives, Revise intersects the change with recorded data and control dependencies, propagates the impact through the partially executed DAG, stops invalid work, preserves unaffected progress, and recomputes only the affected region. Experiments on real coding‑agent traces and LangGraph/LLMCompiler applications show that Revise matches a latest‑version oracle, reduces model calls by up to 56%, and improves service‑level objective goodput under load.

By Ruoling Qi, Xuaner Wu, Penghang Liu, Jian Chen, Yirui Liu
arXiv AI
Oct 16

ClaimScope: Label-Blind Witness Verification for Mathematical Claim Auditing

ClaimScope is a 72‑task benchmark that audits mathematical claims by requiring a verdict, scope, counterexample, and repair, covering areas such as optimizer‑state transport and exact $2 imes2$ linear algebra. The study evaluates Qwen3‑8B using a witness checker that only examines the model’s submitted state, showing that verifier feedback improves counterexample acceptance and safe completions compared to generic feedback. The results demonstrate that separating claim classification from executable evidence can yield gains in witness‑structure correction without necessarily improving broader reasoning.

By Mushir Akhtar