arXiv Computation and Language

Beyond Corrected Actions: Training LLM Agents on Textual Critiques

arXiv Computation and Language

Sep 1

CAST: Critique-Aware Supervision for Training Reliable Long-Horizon Tool-Calling Agents

CAST is a critique‑aware training framework that transforms sparse task outcomes into action‑level supervision for both critique learning and policy optimization. By analyzing agent trajectories, CAST synthesizes structured rationales that explain action validity under partial observability, enabling the creation of richer training data. Fine‑tuned Qwen3‑family models trained with CAST show significant reliability gains, outperforming GPT‑OSS‑120B by over 10% on Retail tasks and improving Telehealth performance by 9% in an out‑of‑domain setting.

By Amir Saeidi, Zehua Zhang, Rishitosh Singh, Naman Ahuja, Vivek Gupta, Ali Payani, Gaowen Liu, Jayanth Srinivasa, Chitta Baral

llms agents fine-tuning benchmarks

arXiv Machine Learning

Oct 8

Training Advisors for LLM Agents from Task Outcomes

arXiv:2610.09858v1 Announce Type: cross Abstract: Large language model agents tackle multi-step tasks by interleaving reasoning and tool calls with observations from the environment. Prior work has s...

By Sergei Polezhaev, Barys Liskavets, Ori Press, Alexander Golubev

llms agents nlp reinforcement-learning benchmarks

arXiv AI

Oct 7

Selective Critique for Cost-Aware LLM Agents in Long-Horizon Decision Making

arXiv:2610.07335v1 Announce Type: cross Abstract: Improving the reliability of large language model (LLM) agents in long-horizon decision-making remains a key challenge. When deployed as autonomous a...

By Heewon Park, Somin Im, Minhae Kwon

llms agents benchmarks

arXiv AI

Jul 24

From Agent Failures to Text Policies: What Works and What Breaks

arXiv:2607. 20668v1 Announce Type: cross Abstract: TextGrad improves language-model systems by revising text from feedback.

By Jaideep Ray, Ankit Goyal

llms agents

arXiv AI

Oct 1

Rep2Skill: Representation-Guided Skill Self-Evolution for LLM Agents

Rep2Skill introduces a representation-guided framework that enables large language model agents to self-evolve their textual skills by analyzing internal representation trajectories from agent rollouts. The method identifies execution turns that deviate from successful dynamics and uses these signals, together with execution contexts, as actionable feedback for targeted skill revision. Experiments with two open-source LLMs across two agent environments demonstrate that Rep2Skill consistently outperforms purely text-based approaches, showing that incorporating internal representations can enhance agent self-improvement.

By Kaixing Zhang, Changming Li, Yingdong Shi, Zheng Zhang, Kaitao Song, Wenjie Shi, Jingang Wang, Kan Ren

llms agents

Hugging Face Trending Papers

Sep 28

Shockingly Simple Self-retrospection Improves Agentic Models Without RL

The paper explores Retrospection-Only Fine-Tuning (ROFT), a method where a language-model agent improves its behavior by generating and training on explanations of its own experiences, without external teachers or reward signals. In software‑engineering tasks with Qwen3.5‑4B, ROFT achieves comparable or better solve rates than GRPO while requiring fewer updates and training time, and can learn from failures alone. Behavioral analysis shows ROFT indirectly assigns credit to actions and can produce shorter, more direct solutions when prompted to focus on direct solutions.

llms agents reinforcement-learning fine-tuning

arXiv AI

Jun 2

Weak Critics Make Strong Learners: On-Policy Critique Distillation for Scalable Oversight

arXiv:2606. 00424v1 Announce Type: new Abstract: As large language models become stronger, weak supervisors may fail to provide reliable labels, preferences, or final judgments for complex outputs, limiting both weak-to-strong generalization and scalable oversight.

By Can Jin, Jiakang Li, Rui Wu, Eddy Zhang, Dimitris N. Metaxas

llms efficiency benchmarks safety

arXiv AI

Oct 1

SkillMaster: Toward Autonomous Skill Mastery in LLM Agents

arXiv:2605.08693v3 Announce Type: replace Abstract: Skills provide an effective mechanism for improving LLM agents on complex tasks, yet in existing agent frameworks, their creation, refinement, and...

By Min Yang, Jinghua Piao, Xu Xia, Xiaochong Lan, Jiaju Chen, Yongshun Gong, Yong Li

llms agents benchmarks

arXiv AI

Jul 14

Agentic-DPO: From Imitation to Agentic Policy Optimization on Expert Trajectories

arXiv:2607. 10601v1 Announce Type: new Abstract: Large Language Model (LLM) agents are commonly trained from expert trajectories using supervised fine-tuning (SFT), which treats multi-turn agent behavior as ordinary text imitation.

By Yixiong Chen, Alan Yuille

llms agents reinforcement-learning fine-tuning

arXiv AI

Jun 26

Joint Learning of Experiential Rules and Policies for Large Language Model Agents

arXiv:2606. 27136v1 Announce Type: new Abstract: For LLM agents in multi-step interactive environments, a key challenge is to make effective use of accumulated interaction experience.

By Shicheng Ye, Chao Yu

llms agents

arXiv AI

Sep 23

When and How Should an Agent Clarify? CIGAsk: Teaching LLMs to Clarify via Counterfactual Information Gain

arXiv:2609.24290v1 Announce Type: new Abstract: Instruction-tuned LLMs faced with underspecified queries often commit to a single interpretation rather than ask for clarification, producing confident...

By Yunxiang Li, Xixin Wu, Helen Meng

llms agents reinforcement-learning benchmarks

arXiv Computation and Language

Sep 1

Co-Evolving Actor-Conditioned Critics for Non-Verifiable Generation

arXiv:2608.30397v1 Announce Type: new Abstract: Natural-language critiques provide supervision beyond scalar rewards for non-verifiable generation, which lacks deterministic verifiers. In critique-gu...

By Jinyoung Kim, Muhammad Khalifa, Lajanugen Logeswaran, Jaekyeom Kim, Moontae Lee, Honglak Lee, Lu Wang

reinforcement-learning

Related stories