arXiv AI
Jun 26
Confidence-Aware Tool Orchestration for Robust Video Understanding
arXiv:2606. 26904v1 Announce Type: cross Abstract: Video reasoning language models implicitly assume that every input frame is equally reliable.
Accuracy and Cost Claims Do Not Survive Re-Execution in Agentic VideoQA
Related stories
arXiv AI
Jul 14
AgentAbstain: Do LLM Agents Know When Not to Act?
arXiv:2607. 10059v1 Announce Type: new Abstract: Agent systems based on large language models (LLMs) are increasingly deployed for autonomous tasks, yet existing evaluations mostly focus on task success rather than whether agents know when to abstain.
arXiv AI
Aug 7
SkillTV-Bench: Benchmarking How Well Judges Perform on Skill-Augmented Agentic Execution
arXiv:2608. 05573v1 Announce Type: new Abstract: LLM agents increasingly execute long-horizon tasks through tool use and environment interaction, shifting evaluation from final-response scoring to verification of complete executions.
arXiv Machine Learning
Aug 28
Finding the Right Evidence: Factor-Guided Coarse-to-Fine Reasoning for Long Videos
The paper introduces PACE, a factor-guided, progressive framework for acquiring evidence in long-video question answering. PACE first indexes clip-level descriptions using question-derived factors, then refines evidence retrieval with contrastive cues derived from candidate answers. On the MMR‑V dataset, PACE achieves 42.6% accuracy and recovers 66.9% of annotated cues, outperforming direct inference and prior agentic baselines, and shows consistent improvements across several long-video benchmarks.
arXiv AI
Jul 21
AEVAL: From Anecdotal to Deterministic Testing for Agentic Skill Workflows
arXiv:2607. 16345v1 Announce Type: cross Abstract: Modern agentic systems increasingly rely on skills: installable packages of natural language and code that teach an LLM agent to perform a domain task.
arXiv AI
Jul 14
Interaction Scaling: Grounding the Third Axis of Test-Time Compute
arXiv:2607. 11598v1 Announce Type: new Abstract: There are two standard ways to spend more compute at test time: let a model reason longer, or sample more attempts and keep one.
arXiv AI
Jul 3
Reasoning effort, not tool access, buys first-try reliability in agentic code generation: an observational study
arXiv:2607. 02436v1 Announce Type: cross Abstract: Agentic coding assistants are increasingly given extra capabilities, such as browser based testing tools and design oriented system prompts, on the assumption that more capability yields better software.
arXiv AI
Aug 19
On the Fragility of Self-Improving Agents: Variance, Task Order, and Underspecification
The paper re‑evaluates memory‑based self‑improving agents by adding multiple runs to measure variance and by randomizing task order. It finds that agent performance is noisy in complex, multi‑step environments and that improvement depends heavily on the sequence of tasks, revealing a hidden curriculum effect. The authors suggest that underspecification of tasks and environments contributes to this fragility and demonstrate that adding detailed rubrics and feedback can partially mitigate performance drops, though gaps remain.
arXiv AI
Aug 3
M3MAD-Bench: Multi-Dimensional Evaluation of Multi-Agent Debate Across Domains and Modalities
arXiv:2601. 02854v2 Announce Type: replace Abstract: As an agent-level reasoning and coordination paradigm, Multi-Agent Debate (MAD) orchestrates multiple agents through structured debate to improve answer quality and support complex reasoning.
arXiv AI
Aug 11
MonitorBench: A Comprehensive Benchmark for Chain-of-Thought Monitorability in Large Language Models
arXiv:2603. 28590v3 Announce Type: replace Abstract: Large language models (LLMs) can generate chains of thought (CoTs) that are not always causally responsible for their final outputs.
arXiv AI
Jun 2
Early Diagnosis of Wasted Computation in Multi-Agent LLM Systems via Failure-Aware Observability
arXiv:2606. 01365v1 Announce Type: new Abstract: Tool-using multi-agent large language model (LLM) systems spend computation through model tokens, tool calls, retries, and code execution before producing an answer.
Hugging Face Trending Papers
Aug 6
SkillTV-Bench: Benchmarking How Well Judges Perform on Skill-Augmented Agentic Execution
LLM agents increasingly execute long-horizon tasks through tool use and environment interaction, shifting evaluation from final-response scoring to verification of complete executions. For skill-augmented agents, verification additionally requires the procedural knowledge encoded in task-time skills, because this knowledge indicates what evidence to inspect and which failures are task-critical.