arXiv AI
Jul 24
From Agent Failures to Text Policies: What Works and What Breaks
arXiv:2607. 20668v1 Announce Type: cross Abstract: TextGrad improves language-model systems by revising text from feedback.
AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systems
Related stories
arXiv Computation and Language
Oct 8
Goldsmith: Gold-Loss-Guided Definition Optimization with an Agentic Annotation Harness
Goldsmith is an agentic pipeline that transforms a small set of expert-annotated calibration examples into a reusable structured annotation definition treated as a trainable textual object. Candidate definitions are evaluated on the same gold examples using an executable structured loss, and a large language model editor revises high-loss failures only when loss decreases. The approach outperforms direct rewriting, OPRO, APE, and PromptBreeder in prompt-optimization tests and enhances downstream annotation across typed span, pair-level relation, and fixed-trigger event-argument tasks when combined with retrieval, score-based routing, and human review.
arXiv Machine Learning
Jun 24
Stabilizing Black-Box Prompt Optimization with Textual Regularization and Signal Aggregation
arXiv:2507. 09839v2 Announce Type: replace Abstract: An increasing number of NLP applications interact with large language models (LLMs) through black-box APIs, making prompt engineering critical for controlling model behavior.
arXiv AI
Sep 18
Rethinking Multi-Agent Collaboration: When More Is Less
The paper examines when multi‑agent collaboration is beneficial versus single‑agent approaches. It finds that collaboration yields systematic advantages mainly in long‑horizon tasks with sparse dependencies, while single agents perform better in tightly coupled, sequential workflows. The authors introduce SAIGE, a lightweight multi‑agent mechanism that models collaboration as a dynamically evolving graph, and show that it balances context efficiency and task performance without always improving outcomes as more agents are added.
arXiv Computation and Language
4d ago
SAGE: Stochastic Prompt Optimization via Agent-Guided Exploration
arXiv:2606.18902v3 Announce Type: replace Abstract: Context engineering has emerged as a primary lever for improving AI systems without parameter updates. Recent work showing that textual gradients d...
arXiv AI
Oct 27
Recursive Semantic Divergence for LLM Agent Consistency
The paper introduces Recursive Semantic Divergence (RSD), an unsupervised, trajectory‑level consistency metric for large language model agents that use tools. RSD recursively measures expected cumulative semantic disagreement between two independent rollouts from the same state, using natural language inference to detect logical inconsistency. When applied as a reward signal during fine‑tuning, RSD significantly reduces variability in task progress across benchmarks and model scales while matching supervised fine‑tuning performance without needing ground‑truth labels.
Hugging Face Trending Papers
Jul 15
AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities
As Large Language Models (LLMs) evolve into autonomous agents, the need for unified evaluation infrastructure becomes critical. However, current evaluation pipelines remain highly fragmented and tightly coupled, hindering reproducibility and causing redundant engineering.
Hugging Face Trending Papers
Jun 17
SAGE: Stochastic Prompt Optimization via Agent-Guided Exploration
Context engineering has emerged as a primary lever for improving AI systems without parameter updates. Recent work showing that textual gradients do not function as real gradients motivates treating automatic prompt optimization (APO) as black-box search.
Hugging Face Trending Papers
Sep 17
Rethinking Multi-Agent Collaboration: When More Is Less
The paper examines when multi‑agent collaboration is truly beneficial as large language models grow more capable. It finds that multi‑agent systems yield systematic advantages mainly for long‑horizon tasks with sparse dependencies, while single‑agent approaches excel in tightly coupled, sequential workflows. The authors introduce SAIGE, a lightweight, graph‑based collaboration framework that balances context efficiency and performance, demonstrating that adding more agents or deeper recursion does not always improve outcomes.
arXiv AI
Oct 21
MaD-RSI: Model-as-Data for Recursive Self-Improvement of Agent Harnesses
arXiv:2610.28201v1 Announce Type: new Abstract: Recursive self-improvement (RSI) of agent harnesses has emerged as an effective approach to enhancing the capabilities of large language model (LLM) ag...
arXiv AI
Oct 28
When the Critic Falls Behind: Co-Evolving Rubrics and Harnesses for Continual Agent Self-Improvement
The paper introduces RuHarness, a framework that co‑evolves rubrics and harnesses to enable continual self‑improvement of autonomous agents that rely on large language models. RuHarness automatically discovers and reassesses behavioral criteria from execution trajectories, prioritizes unresolved weaknesses, and uses these insights to guide targeted harness updates while keeping the task objective unchanged. Experiments across varied agent tasks and backbone models demonstrate that RuHarness outperforms existing harness‑evolution baselines, especially on unseen held‑out tasks, and that its adaptive rubric prioritization supports sustained improvement and stable generalization over successive evolution rounds.
arXiv AI
3d ago
AgentSelect: Benchmark for Narrative Query-to-Agent Recommendation
AgentSelect is a benchmark that reframes agent selection as narrative query‑to‑agent recommendation, providing a unified dataset of 111,179 queries, 107,721 deployable agents, and 251,103 interaction records drawn from over 40 sources. It converts heterogeneous evaluation artifacts into positive‑only interaction data, enabling systematic study of how to recommend end‑to‑end agent configurations that combine a backbone model with a toolkit. Analyses show a shift from dense head reuse to long‑tail supervision, revealing that content‑aware capability matching is essential and that models trained on AgentSelect transfer to real‑world agent marketplaces with consistent gains. whyItMatters":"AgentSelect supplies the first unified data and evaluation infrastructure for agent recommendation, establishing a reproducible foundation to study and accelerate the emerging agent ecosystem."