arXiv AI
Jul 9
AgentLens: Production-Assessed Trajectory Reviews for Coding Agent Evaluation
arXiv:2607. 06624v1 Announce Type: new Abstract: We present AgentLens, a production-assessed benchmark for interactive code agents.
Trace & Evaluate your Agent with Arize Phoenix
Related stories
arXiv AI
Jun 8
TRACE: Trajectory Reasoning through Adaptive Cross-Step Evidence Aggregation for LLM Agents
arXiv:2606. 07054v1 Announce Type: cross Abstract: Autonomous LLM agents can pursue hidden malicious objectives through sequences of individually benign actions, making sabotage difficult to detect using standard trajectory-level monitoring.
arXiv AI
Jun 19
AgentFinVQA: A Deployable Multi-Agent Pipeline for Auditable Financial Chart QA
arXiv:2606. 19782v1 Announce Type: new Abstract: Financial chart question answering in regulated settings demands more than accuracy: practitioners must know which answers to trust before acting on them, and many institutions cannot send client data to external model providers.
arXiv AI
Jul 9
Progressive Crystallization: Turning Agent Exploration into Deterministic, Lower-Cost Workflows in Production
arXiv:2607. 07052v1 Announce Type: cross Abstract: AI agents deployed for IT operations are typically permanent cost centers because every execution requires full LLM inference, even for previously solved problems.
Mistral AI
May 27, 2025
Build AI agents with the Mistral Agents API
arXiv AI
Aug 25
ClawProBench: Trace-Aware Evaluation of AI Agents with Runtime Coverage and Frozen Workplace-Style Holdouts
arXiv:2608.22510v1 Announce Type: new Abstract: Agent benchmarks often evaluate only final answers even when agents run on stateful runtimes. We argue this under-specifies what is being evaluated: th...
arXiv AI
Jul 22
AgentTrails: Towards Trust and Reuse for Agentic Tasks
arXiv:2607. 18816v1 Announce Type: cross Abstract: LLM-powered agents increasingly tackle complex tasks by invoking tools, querying databases, executing code, and manipulating intermediate artifacts.
arXiv AI
Jun 16
When Agent Automation Becomes Profitable: Quantifying and Insuring Autonomous AI Risk through Trace-Economic Underwriting
arXiv:2606. 16465v1 Announce Type: new Abstract: AI agents can now take irreversible actions in operational systems, but agent-caused losses are still not clearly assigned, priced, or transferred.
arXiv AI
Jun 6
Do More Agents Help? Controlled and Protocol-Aligned Evaluation of LLM Agent Workflows
arXiv:2606. 05670v1 Announce Type: new Abstract: Does adding more agents help an LLM workflow once compared systems share the same benchmark loader, tool access, answer contract, usage accounting, and trajectory logging?
arXiv AI
Jul 10
Context Graphs for Proactive Enterprise Agents
arXiv:2607. 07721v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) and agentic frameworks have advanced enterprise AI considerably, yet agents remain fundamentally reactive: they wait for a human query before acting.
arXiv AI
Jul 7
TRACE: Capability-Targeted Agentic Training
arXiv:2604. 05336v2 Announce Type: replace Abstract: Models often fail to complete agentic tasks because they lack core capabilities required by the target environment.
arXiv AI
Aug 7
TRAJDEBUG: Tracing Error Lifecycle to Identify Critical Failures in Long-Horizon Agent Trajectories
arXiv:2608. 06346v1 Announce Type: new Abstract: LLM-based agentic systems have shown remarkable capabilities in complex domains, while suffering from cascading errors and difficulty in debugging.