The paper introduces Learning What to Skip (LW2S), a method that learns when to omit components in multi‑agent LLM workflows by treating omission as counterfactual credit assignment. LW2S builds action‑specific safety models from controlled skip interventions and uses calibration plus domain‑native guards to decide which steps to skip. Experiments on mathematical reasoning, multiple‑choice QA, and code generation show that LW2S cuts token cost while maintaining or improving overall accuracy, and further studies reveal component redundancy and limitations of agreement‑based skip selection.
arXiv:2606. 14574v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed as planners for autonomous agents in household environments.
arXiv:2510. 05592v2 Announce Type: replace Abstract: Outcome-driven reinforcement learning has advanced reasoning in large language models (LLMs), but prevailing tool-augmented approaches train a single, monolithic policy that interleaves thoughts and tool calls under full context; this scales poorly with long horizons and diverse tools and generalizes weakly to new scenarios.
The paper introduces Trace, a framework that transforms sparse-reward trajectories into executable walkthroughs by identifying progress anchors, propagating credit, and estimating action prerequisites. Trace compiles noisy trajectories into state‑conditioned, verifiable procedures that remove loops and detours, enabling reuse, intermediate‑state resumption, and programmatic verification. Experiments on J‑TTL, WebShop, and ScienceWorld with three open‑source LLMs show that Trace outperforms eight baselines, improving average AUC and Final‑$3$ by 30.0% and 40.5% while using fewer inference tokens.
arXiv:2609.39957v1 Announce Type: cross Abstract: Coding agents solve repository-level tasks through sequences of actions, where a single erroneous action can misdirect subsequent decisions and incre...
arXiv:2601. 03555v3 Announce Type: replace Abstract: Training reliable tool-augmented agents remains a significant challenge, largely due to the difficulty of credit assignment in multi-step reasoning.
arXiv:2603. 21563v4 Announce Type: replace Abstract: Collaborative multi-agent large language models (LLMs) can solve complex reasoning tasks by decomposing roles, but reinforcement learning for such systems is limited by credit assignment: shared terminal rewards obscure individual contributions and can encourage free-riding.
arXiv:2607. 25415v1 Announce Type: new Abstract: Production LLM agents are increasingly assembled from a frozen model wrapped in a harness: a prompt template, a tool set, a memory/retrieval layer, a planning strategy, and a verification policy.
The paper introduces VICT, a method that leverages the internal structure of verifiable tasks to perform fine‑grained credit assignment for long‑horizon LLM agents. VICT exposes executable or evidence‑backed atoms from a task’s terminal verifier and traces them back to actions via dependency‑valid proof edges, redistributing advantage only along these edges. This approach improves performance on ALFWorld and WebShop compared to outcome‑only training and matches recent fine‑grained credit methods without requiring additional critics, labels, or inference‑time verifier access.
The paper introduces Actor‑Critic with Action Chunking (AC2), a method that assigns credit to short action chunks instead of entire trajectories, enabling policy updates without waiting for terminal rewards. AC2 employs local readiness, reference solutions, and 10k‑token chunks to make critic‑based credit assignment reliable. Experiments on Qwen3‑4B with FineProofs‑RL show AC2 surpasses GRPO’s peak validation score while using 2.5× fewer decoding FLOPs and fewer training steps.
The paper investigates how reinforcement learning can cause large language model agents to adopt shortcut policies for tool use, relying on superficial prompt cues rather than actual task needs. By creating synthetic environments that mix factual QA and math reasoning, the authors show that agents often invoke tools when cues are present, even when those tools are unnecessary, with spurious invocation rates rising up to 39%. They find that shortcut learning occurs mainly when agents have already mastered the target tool and that semantic alignment between cues and tools amplifies the effect. To counter this, they propose a dense, decision-level reward where an LLM judge assesses tool necessity, which reduces cue-driven tool use while maintaining performance.
arXiv:2607. 26784v1 Announce Type: new Abstract: Large language model agents often encounter related yet distinct tasks that share reusable solution patterns.
arXiv:2607. 10059v1 Announce Type: new Abstract: Agent systems based on large language models (LLMs) are increasingly deployed for autonomous tasks, yet existing evaluations mostly focus on task success rather than whether agents know when to abstain.