arXiv AI
Jul 7
CausalGame: Benchmarking Causal Thinking of LLM Agents in Games
arXiv:2607. 04293v1 Announce Type: cross Abstract: Building AI Scientist agents with Large Language Models (LLMs) has recently attracted growing attention.
ACED-Bench: Evaluating How LLM Agents Acquire and Act on Causal Evidence
Related stories
arXiv AI
Jul 10
CausalDS: Benchmarking Causal Reasoning in Data-Science Agents
arXiv:2607. 08093v1 Announce Type: new Abstract: Large language models (LLMs) increasingly act as integrated data-science agents, combining abstract reasoning with advanced tool use.
Hugging Face Trending Papers
Jul 9
CausalDS: Benchmarking Causal Reasoning in Data-Science Agents
Large language models (LLMs) increasingly act as integrated data-science agents, combining abstract reasoning with advanced tool use. Yet the relevant benchmark landscape largely divides into symbolic causal reasoning benchmarks without realistic data analysis or data analysis benchmarks without a principled causal data-generating structure.
arXiv AI
3d ago
PANINI-DR: Controlled Deep Research through Generative Semantic Workspaces
arXiv:2610.15208v1 Announce Type: cross Abstract: Deep research requires connecting multiple steps of search, reasoning, and computation, with intermediate findings guiding what to investigate next....
arXiv AI
Jun 16
VeriGraph: Towards Verifiable Data-Analytic Agents
arXiv:2606. 16603v1 Announce Type: cross Abstract: LLM-based agents have demonstrated strong capabilities in data-intensive analytical tasks, yet their outputs are rarely verifiable: a reliance on linear text trajectories makes their reasoning difficult to audit.
arXiv AI
2d ago
AlphaDiana: A Measurement System for Agents in Verifiable Reasoning Tasks
AlphaDiana is a measurement system designed to evaluate agents in verifiable reasoning tasks by providing modular interfaces for datasets, agents, environments, and scorers. It maintains harness-native control loops while separately managing file reuse and memory retention, enabling both macro-level analyses of entire agents and micro-level interventions on components such as tools, skills, and memory. Experiments on open-source models and harnesses across multiple benchmarks reveal that harnesses and component interventions do not consistently improve reasoning accuracy, and that token usage metrics must be interpreted alongside correctness.
arXiv AI
Jul 14
FIRE-Bench: Evaluating AI Agents on the Rediscovery of Scientific Insights
arXiv:2602. 02905v2 Announce Type: replace Abstract: Autonomous agents powered by large language models (LLMs) promise to accelerate scientific discovery end-to-end, but rigorously evaluating their capacity for verifiable discovery remains a central challenge.
arXiv AI
Oct 9
Proof-of-Use: Mitigating Tool-Call Hacking in Deep Research Agents
The paper introduces Proof-of-Use (PoU), a reinforcement learning framework designed to prevent tool‑call hacking in deep research agents. PoU enforces evidence grounding by requiring agents to cite normalized evidence identifiers and uses a multi‑objective reward system that includes progressive process rewards, an answer‑support alignment reward, and a curriculum‑style adaptive reward mixing. Experiments demonstrate that PoU effectively mitigates tool‑call hacking and promotes robust, adaptive tool usage even under domain and tool shifts.
Hugging Face Trending Papers
Aug 31
Learning to Evaluate Before Improving: Automatic Rubric Induction for Automatic Research Agents
Autonomous scientific research agents are increasingly applied to end-to-end scientific workflows, including literature review, data analysis, experimentation, and report generation. However, open-end...
arXiv AI
Jun 11
Can AI Agents Synthesize Scientific Conclusions?
arXiv:2606. 11337v1 Announce Type: new Abstract: Scientific AI agents increasingly retrieve evidence, reason across sources, and synthesize conclusions used in consequential decisions.
arXiv AI
Sep 1
Learning to Evaluate Before Improving: Automatic Rubric Induction for Automatic Research Agents
arXiv:2608.31076v1 Announce Type: cross Abstract: Autonomous scientific research agents are increasingly applied to end-to-end scientific workflows, including literature review, data analysis, experi...
arXiv AI
Jun 12
(Human) Attention Is (Still) All You Need: Human oversight makes AI-assisted social science reliable
arXiv:2606. 12848v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for tasks once reserved for trained researchers, including hypothesis generation, specification choice, and drafting conclusions.