AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,743 stories · RSS feed

arXiv AI
Oct 5

MOF-VERIFY: A Failure-Aware Agentic Harness for MOF Hypothesis Verification

MOF-VERIFY is a failure-aware agentic harness designed to improve hypothesis verification for metal‑organic frameworks (MOFs). It introduces a diagnostic benchmark with four task families—structural grounding, synthesis‑condition verification, evidence‑sufficiency verification, and MLIP‑based computational verification—to pinpoint failures in knowledge access, evidence acquisition, and reasoning. Guided by these diagnostics, MOF‑Verify addresses structural, literature, evidence‑sufficiency, and computational bottlenecks, achieving significant performance gains over direct inference and retrieval‑based baselines across multiple large language models.

By Donghyun Lee, Taehoon Lee, Geonhee Ahn, Jieun Kim, Jihyun Park, Suyeon Cho, Yoona Kim, Chaerim Shin, Hoi Ri Moon, Jonggeol Na, Sukho Hong, Jihwan Oh, Soo Kyung Kim
arXiv AI
Oct 5

ULTRADISCOVERY: Abductive Exploration in an Interconnected, Epistemically Open Universe

ULTRADISCOVERY is a benchmark that tests abductive exploration in an epistemically open, interconnected universe. It features five domains where agents must revise an initial theory and predict outcomes of unseen cross‑domain interventions, with a $2 imes 2$ design manipulating representation openness and evidence alignment. Experiments show that agents struggle to compose evidence into transferable representations, especially when the representation is open, and that even with aids exact predictions are rare without large budgets.

By Weihan Li, Tianshi Zheng, Yangqiu Song, Ginny Y. Wong, Simon See
Towards Data Science
Oct 4

How to Govern AI Agents

The article "How to Govern AI Agents" discusses strategies for overseeing a single AI agent and expanding those governance practices to manage a fleet of agents. It explores the challenges and solutions involved in scaling control mechanisms as the number of agents increases. The piece highlights practical approaches for ensuring reliable and ethical operation across multiple AI systems.

By Amber Roberts
Simon Willison
Oct 3

We're going to need default hard budget caps on pretty much everything

Simon Willison argues that pay‑by‑usage services and APIs should implement default hard budget caps that stop usage and return errors once a set monthly limit is reached, rather than merely sending warning emails. He contends that coding agents and personal agents often trigger costly API calls, and hard caps prevent unexpected large bills. Willison calls for a clear opt‑in checkbox to remove the cap, and notes that AWS has recently introduced spending limits to address this issue.

Towards Data Science
Oct 3

Measuring the Creativity Potential of LLM Agents

The article explores whether large language model (LLM) agents can discover new ideas by examining their creative potential. It frames this inquiry through the lens of creativity, aiming to assess how LLM agents might generate novel insights or solutions. The discussion is presented as a post on Towards Data Science.

By Shitanshu Bhushan
arXiv Machine Learning
Oct 3

Contingent Exposure Routing for Financial AI: Outage Risk and the Cost of Indivisible Decisions

The paper introduces a framework for routing financial decisions during system outages, using a local market‑impact response matrix to analyze expected squared price displacement. It shows that shared backups can create a concentration floor as primary endpoints increase, while balanced fallback risk diminishes with more surviving endpoints. For indivisible decisions, the authors derive exact second moments of randomized routing, propose conditional‑expectation rounding and a swap procedure, and demonstrate risk reductions of 6.57%–10.53% in synthetic tests and 3.30% in real API replay scenarios.

By Shivam Gupta
arXiv Machine Learning
Oct 3

Make Sparse Rewards Count: Density-Aware Reward Aggregation for Multi-Reward RL

The paper introduces Density-Aware Reward Aggregation (DARA), a method that adjusts reward weighting in multi-reward reinforcement learning based on the density of active rewards within rollout batches. By applying an inverse-square-root density correction, DARA gives more weight to less frequently active rewards, enabling faster learning of targeted behaviors. Experiments on tool calling and mathematical reasoning demonstrate that DARA achieves comparable final performance while reducing training steps by up to 26% and 65% respectively.

By Tong Zheng, Skylar Zhai, Zhan Cheng, TianMing Sha, Youling Huang, Shuo Zhou, Shaotong Qi, Jingcheng Liang, Xuwei Ding, Pengcheng Xu
arXiv Machine Learning
Oct 3

How Much Can Language Models Gain from Test-Time Computation?

The paper investigates how test‑time computation can enhance language models and at what cost, introducing the SELF‑POT benchmark to evaluate this across competition mathematics, competitive programming, and agentic workflows. SELF‑POT separates candidate coverage from final accuracy, tracks correctness transitions under revision, and measures protocol completion alongside task success. Using a unified budget rule, the study compares direct inference, parallel sampling, and self‑revision across five low‑cost reasoning models, revealing that selection rules and failure handling significantly influence gains and cost savings.

By Bangji Yang, Jingyuan Li, Jiajun Fan, Yi Evie Zhang, Ruihan Guo, Hongba Ma, Neil He, Chumeng Liang, Qinglong Zheng, Zhanghan Ni, Ge Liu