arXiv Machine Learning
Jun 25
Reclaim Evaluation: A Lossy Memory Is Worse Than an Empty One
arXiv:2606. 25449v1 Announce Type: cross Abstract: A language model's memory can be worse than having no memory at all.
Fidelity Collapses with Table Size, Not Turn Count: Delivery Architecture in a Deployed Tool-Using Agent
The study compares two delivery architectures for a multi‑turn agent that uses a tool to retrieve structured records. In the SYNTHESIS architecture, all records are fed into the model’s context, leading to a dramatic drop in accuracy as the table grows beyond about 74 rows. The BYPASS architecture keeps records in session state and delivers them via a typed side channel, maintaining high accuracy (92.3%) regardless of table size across four large language models.
Related stories
Hugging Face Trending Papers
Jun 24
Reclaim Evaluation: A Lossy Memory Is Worse Than an Empty One
A language model's memory can be worse than having no memory at all. Give a model a memory that kept a wrong conclusion but dropped the work behind it, and it emits that stale value as a confident answer; give the same model an empty memory and it abstains.
arXiv AI
Sep 2
REVISE: Validity-Guided Recovery for Online Revisions in Agent Workflows
The paper introduces “Revise”, a runtime system that performs validity-guided, fine-grained recovery for online revisions in structured agent workflows. When a revision arrives, Revise intersects the change with recorded data and control dependencies, propagates the impact through the partially executed DAG, stops invalid work, preserves unaffected progress, and recomputes only the affected region. Experiments on real coding‑agent traces and LangGraph/LLMCompiler applications show that Revise matches a latest‑version oracle, reduces model calls by up to 56%, and improves service‑level objective goodput under load.
arXiv AI
Sep 2
Invalidation Contracts for Cross-Episode Agent Memory
The paper proposes invalidation contracts to manage cached recovery suggestions in LLM agents, attaching version stamps and cacheability hints to each suggestion so stale entries can be evicted without trial and error. The protocol separates realized savings into validity (protocol‑dependent) and compliance (planner‑dependent), showing that row‑level invalidation can significantly improve first‑try compliance and recover a substantial portion of token costs across multiple models, while table‑level invalidation can be detrimental. The study evaluates the approach across seven models, three serving paths, two domains, and about 9,400 episodes, demonstrating deterministic validity and high eviction precision.
arXiv AI
Oct 16
Trajectory-Level Speculation with Semantic Verification for Tool-Using Agents
The paper introduces SpecAgent, a protocol that lets a strong language model act as a reviewer rather than a generator for tool‑using agents. A weaker model drafts steps, executing read‑only tool calls to ground its proposals, while the strong model verifies the draft in a single prefill pass, committing the longest valid prefix and repairing any rejected step. Experiments on benchmarks show SpecAgent achieves near‑strong‑model performance while cutting memory traffic, wall time, and token generation.
arXiv AI
Aug 25
Right-Sizing LLM-Agent Decomposition in VAT Determination: A Pilot Controlled Sweep
The study evaluates how to best split tasks among large‑language‑model agents for cross‑border VAT determination, comparing one broad agent to configurations ranging from one to five narrow agents. Across 4,400 runs—including token‑matched and failure‑injection scenarios—the intermediate configurations achieved the highest accuracy but did not surpass the fine‑endpoint benchmark, leaving the optimal decomposition hypothesis unconfirmed. The pilot provides a preregistered heuristic for right‑sizing decomposition, along with an oracle, dataset, and analysis pipeline.
arXiv Machine Learning
Sep 14
What Drives Recovery in Agentic Text-to-Cypher? LAST-CQ: An LLM Agent Self-Refinement Framework
The paper introduces LAST-CQ, a five-agent, training‑free, execution‑grounded framework for Text‑to‑Cypher that evaluates which components of an agentic pipeline contribute most to performance. Experiments on 2,471 live‑database queries across six backbones show that removing correction reduces execution‑BLEU by 3.1–12.3%, while substituting schema‑grounded feedback with raw error strings has negligible impact. Parallel sampling degrades quality by 10–11%, whereas failure detection and retry routing recover 91.7% of initially failed queries, highlighting that simple failure handling is more effective than sophisticated feedback or increased sampling.
arXiv AI
Aug 26
ACE: A Self-Correcting Agentic Canvas Editor for Multi-Slide Presentation Automation
ACE is a self‑correcting agentic canvas editor that operates on a hierarchical scene‑graph rather than flat document formats, enabling reliable multi‑slide presentation automation. It pairs a presentation‑specialized action space of 98 tools with CARE, a content‑aware router that reduces input tokens by about 89%, and a ground‑truth‑free instruction‑following judge that feeds natural‑language critiques back into the agent for self‑correction. In benchmarks, ACE outperforms a comparable agentic HTML pipeline on instruction following (4.23 vs. 3.81), runs 1.75× faster, costs 44% less, and is preferred by 58.7% of blind raters, with 81% favoring the self‑corrected output.
arXiv Machine Learning
Aug 11
The Replay Gap: Static Evaluation of Model Switching in LLM Agents Scores the Wrong World
arXiv:2608. 08239v1 Announce Type: new Abstract: LLM routers promise efficiency by matching each request to the cheapest adequate model, and are increasingly applied per step inside multi-step agents.
arXiv Machine Learning
Aug 12
The Evaluation Protocol Determines the Result: An Independent Reproduction of LeWorldModel on TwoRoom
arXiv:2608. 10145v1 Announce Type: new Abstract: LeWorldModel trains a latent world model with a prediction loss and a single anti-collapse regulariser, and reports approximately 87% of goals reached on TwoRoom, its simplest diagnostic environment.
arXiv AI
Oct 7
One Step at a Time: Trading LLM Autonomy for Process Predictability
arXiv:2610.07817v1 Announce Type: cross Abstract: Organizations automating operational processes need more than a correct outcome: they need to predict how a process will run, know which one actually...
arXiv AI
Aug 5
TraceCompiler: Skill-Guided Mining and Compilation of LLM Agent Traces into Mostly Deterministic Workflows
arXiv:2608. 02680v1 Announce Type: cross Abstract: Tool-using language-model agents repeatedly rediscover procedures they have already executed, producing traces that mix reusable structure with retries, exploration, accidental ordering, and repeated lookups.