AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,805 stories · RSS feed

arXiv AI
Aug 24

Categorical AI phenomenology: A first-person approach

The paper proposes a phenomenology-first framework for artificial consciousness, redefining consciousness as the subjective experience produced through an agent’s interface with the world. It models first-person structures mathematically using categories derived from Q-networks, which serve as relational interfaces encoding agent‑world interactions. This approach aligns with 4E cognition theories—enactive, embedded, and extended—and offers a rigorous, categorical account of how computational systems can embed information processing into phenomenological structure.

By Robert Prentner
arXiv AI
Aug 24

PrimeAgentOrchestrator: Memory-Primed Agent Spawning for Personal AI Infrastructure

PrimeAgentOrchestrator (PAO) is a system that spawns new instances of Claude Code, Anthropic’s terminal-based coding agent, pre‑loaded with relevant memories from a user’s personal databases. PAO queries a PostgreSQL entity‑observation database and a Cloudflare Worker semantic search index in parallel, fuses the results, and injects the compiled briefing into the agent’s filesystem to exploit its auto‑read configuration. The paper reports four months of deployment, detailing three generations of context delivery mechanisms, the failure modes that prompted each redesign, and the engineering tradeoffs involved in integrating heterogeneous memory systems instead of building a unified one.

By Myron Koch (Peak Summit Labs)
arXiv AI
Aug 24

SDAD: Spec-Driven Agentic Development for the AI-Native SDLC

The paper introduces Spec-Driven Agentic Development (SDAD), a framework that leverages large language models to ingest extensive functional requirement documents and repository context in a single workflow, turning specification quality into the engine for autonomous software delivery. SDAD blends disciplined upfront formalisation with rapid implementation, encompassing intent capture, machine‑readable specifications, agentic synthesis, and multi‑agent verification with human sign‑off. It positions AI‑code as a fourth production paradigm, compares it to traditional Waterfall and Agile approaches, and extends the model to team role evolution, quantitative governance metrics, and a staged migration blueprint for practical adoption.

By Vu Hung Nguyen, Thanh Nguyen
arXiv AI
Aug 24

Personalized Privacy Control in LLMs via Attention Head Intervention

The paper introduces the concept of personalized privacy for large language models, allowing user‑specific disclosure preferences to guide information sharing. It presents P3Bench, a benchmark that extends contextual privacy policies with personalized rules, and shows that existing prompt‑based methods often ignore these policies. To improve compliance, the authors propose “Repair”, an inference‑time attention head intervention that aligns model responses with user‑specific privacy rules.

By Junseok Kim, Nakyeong Yang, Kyomin Jung
arXiv Machine Learning
Aug 24

Reinforcing Multi-Turn Reasoning in LLM Agents via Fine-Grained Reward Structure and Credit Assignment

The paper explores how dense, turn-level reward structures can improve reinforcement learning for large language model agents in multi-turn tasks. It introduces three reward granularity types—terminal, delayed, and per-turn—and adapts Group Relative Policy Optimization and Proximal Policy Optimization to each. Experiments on search and game agents show that per-turn rewards consistently yield better training dynamics, faster convergence, and higher answer correctness compared to sparse terminal or delayed rewards.

By Quan Wei, Siliang Zeng, Chenliang Li, Zhongruo Wang, William Brown, Oana Frunza, Wei Deng, Anderson Schneider, Yuriy Nevmyvaka, Yang Katie Zhao, Alfredo Garcia, Mingyi Hong
arXiv AI
Aug 24

An LLM agent for end-to-end computational materials discovery

MAESTRO is a large language model agent that automates the full screening pipeline for metal‑organic frameworks (MOFs). It parses extensive MOF literature, links publications to crystal structures, curates a computation‑ready database, and then applies a progressively more expensive computational strategy to identify promising candidates. The identified materials for wet flue gas separation come from unrelated studies, demonstrating the agent’s ability to uncover high‑performance materials across domains.

By Chen Yuntong, Huang Ju, Liu Yu, Zhao Dan, Sun Mingqi, Ju Chentian, Liu Yanbing, Huang Lijiang, Zhao Guobin
arXiv AI
Aug 24

EditPPT: Faithful Long-Deck Slide Editing via Structured Tool-Using Multi-Agent with Dual-Modal Validators

EditPPT is a multi‑agent framework that turns slide editing into a constrained tool‑selection task, using the native PowerPoint COM interface to perform localized shape‑level operations. By separating validation across modalities, its dual‑modal validators assess both instruction fidelity and visual quality, achieving high execution and accuracy rates even on long decks. The authors also introduce DeckEdit‑Bench, a benchmark of 28 human‑authored decks with 582 slides and 183 editing prompts across varying deck lengths.

By Jiheon Kim, Kyudan Jung, Jaegul Choo
arXiv AI
Aug 24

ReFrame: Evidence-Guided Test-Time Safety Alignment in Multimodal Large Language Models

ReFrame is a training‑free framework that enhances safety alignment for multimodal large language models at test time. It uses two lightweight agents: one generates risk and utility evidence, and the other rewrites prompts and routes images to create a safe proxy before invoking the deployed MLLM. Experiments show that ReFrame improves jailbreak defense, safety awareness, and reduces over‑sensitivity while maintaining multimodal utility.

By Wenzheng Jiang, Xuankun Rong, Yuanzhao Zhai, Dawei Feng, Huaimin Wang
arXiv AI
Aug 24

Don't Solve, Just Compare: Tiny Advisors for Runtime Intervention in LLM Agents

The paper introduces COTA, a lightweight comparison-only framework that enables runtime intervention in large language model agents by using a tiny comparator to evaluate whether alternative continuations outperform the agent’s current proposal. Instead of generating full solutions, the comparator only judges sampled alternatives, and repeated comparisons decide when to intervene. Experiments on WebShop, ALFWorld, and tau^3-Retail show that COTA improves performance across nine settings and outperforms existing baselines, demonstrating that constructive intervention can be effective even with a much weaker auxiliary model.

By Yanze Jiang, Mingxuan Li, Yuhao Wang, Shengfang Zhai, Jiaheng Zhang
arXiv AI
Aug 24

Belief Without Behavior: Measuring the Translation of Theory of Mind into Coordinated Social Action in Vision-Language Models

The paper introduces MOSAIC, a benchmark that tests how vision‑language models translate theory‑of‑mind (ToM) reasoning into coordinated social actions across verbal and nonverbal channels. Across 200 trials per model, 13 models—including 11 VLMs—failed to align their behaviors with ToM constraints, revealing bottlenecks in generating coherent nonverbal signals and interpreting others’ actions. A structured model with an explicit ToM module, PCM‑LLM, succeeded in all conditions, indicating that belief‑action coupling is crucial for these tasks.

By Tonglin Yan, Gregoire Sergeant-Perthuis, David Rudrauf
arXiv AI
Aug 24

No Judgment Without a Reason: Counterfactual Receipts for Versioned AI Evaluators

The paper introduces a framework for evaluating AI systems that not only checks final labels but also tracks the reasoning behind them through three core sources—grounds, norms, and authority—forming an eight-cell counterfactual judgment cube. It defines minimal source replacement sets, called judgment receipts, to explain changes in verdicts and provides certification cost bounds for black-box evaluators. The authors present ReasonBench, a benchmark with 19,520 cases, and demonstrate that while high standard accuracy can mask robustness issues, receipt accuracy reveals significant gaps in reasoning consistency across different models.

By Ye Chen, Weining Zhang
arXiv AI
Aug 24

Volumetric Radiology AI in the Era of Multimodal Large Language Models

The article reviews how multimodal large language models (MLLMs) are expanding radiology AI beyond image‑specific tasks to multimodal reasoning, yet volumetric radiology poses a representational challenge because clinical interpretation needs full 3‑D spatial context and quantitative data. It surveys over 200 studies, categorizing advances in volumetric representation, multimodal understanding, and agentic orchestration, and introduces a Claim‑Design‑Validation framework to align technical, workflow, and clinical claims. The review emphasizes that native volumetric modeling and agentic capabilities must match spatial, quantitative, contextual, and workflow demands, and that clinical credibility hinges on faithful 3‑D representation, traceable behavior, proper validation, and defined human oversight.

By Zanting Ye, Shengyuan Liu, Xin Liu, Chenhui Wang, Zhisong Wang, Jiashuai Liu, Zipei Wang, Cheng Wang, Wentao Pan, Mengjie Fang, Di Dong, Mohammad Salmanpour, Arman Rahmim, Yu Gu, Yong Xia, Hongming Shan, Yixuan Yuan, Yefeng Zheng, Lijun Lu
arXiv AI
Aug 24

Investigating Target Class Influence on Neural Network Compressibility for Energy-Autonomous Avian Monitoring

The paper explores how the number of bird species (target classes) affects the compressibility of neural networks for passive acoustic monitoring on microcontroller units (MCUs). By training and compressing models with varying class counts, the authors show that significant compression can be achieved with minimal performance loss. They also benchmark different hardware platforms and assess the feasibility of deploying energy‑autonomous monitoring devices.

By Nina Brolich, Simon Geis, Maximilian Kasper, Alexander Barnhill, Axel Plinge, Dominik Seu{\ss}
arXiv Machine Learning
Aug 24

Metag: A dataset to build agentic meta-reviewing capabilities

arXiv:2608.20488v1 Announce Type: new Abstract: AI tools increasingly support tasks across the scientific research cycle, from experiment design and manuscript preparation to peer review. At the same...

By Anirudh Sundar, Min Chen, Divya Tadimeti, Gemma Zhang, Alice Li, Nigel Boachie Kumankumah, Pavan Uttej Ravva, Sadid Hasan, Somya Chatterjee, Pruthvi Prakash Navada, Xiao Wang, Yue Kang, Sulaiman Vesal, Larry Heck
arXiv AI
Aug 24

PhotoBench: Beyond Visual Matching Towards Personalized Intent-Driven Photo Retrieval

PhotoBench is a new benchmark built from authentic personal photo albums that moves beyond simple visual matching to focus on personalized, intent-driven retrieval. It incorporates a multi-source profiling framework that combines visual semantics, spatial‑temporal metadata, social identity, and temporal events to generate complex queries reflecting users’ life trajectories. Evaluation on PhotoBench reveals two key limitations: a modality gap where unified embedding models fail on non‑visual constraints, and a source fusion paradox where agentic systems struggle with tool orchestration.

By Tianyi Xu, Rong Shan, Junjie Wu, Jiadeng Huang, Teng Wang, Jiachen Zhu, Wenteng Chen, Minxin Tu, Quantao Dou, Zhaoxiang Wang, Changwang Zhang, Weinan Zhang, Jun Wang, Jianghao Lin
arXiv Machine Learning
Aug 24

TH-GNN: Heterogeneous Temporal Graph Neural Networks for LLM-Agent Shilling Attack Detection

TH-GNN is a heterogeneous temporal graph neural network designed to detect shilling attacks generated by large language model (LLM) agents. It combines a two‑layer Heterogeneous Graph Transformer with per‑type and per‑relation attention, learnable sinusoidal temporal encodings, cross‑modal attention that fuses user embeddings with frozen RoBERTa representations of reviews and item descriptions, and a GRU that models log inter‑arrival times. Across five attack families and four benchmark datasets, TH‑GNN achieves a grand‑mean F1 score of 0.870, surpassing the best text‑only baseline on Agent4SR attacks by 10.9 percentage points and 11.5 percentage points at the lowest injection rate.

By Shivam Swarup, Divya Prakash Shrivastava, Rakesh Thakur
arXiv Machine Learning
Aug 24

AudioWorldSim: Realistic Binaural Audio Datasets For World Models

AudioWorldSim is an open‑source platform that generates realistic binaural audio datasets for training and evaluating audio‑based machine learning models, especially world models. It extends Meta’s SoundSpaces 2.0 by automating random agent navigation and correcting continuous sound composition. The project is publicly available on GitHub to support reproducibility in research.

By Luis Vitor Zerkowski, Luiz Velho
arXiv Machine Learning
Aug 24

Smart Exploration in Reinforcement Learning using Bounded Uncertainty Models

The paper introduces BUMEX, a reinforcement learning exploration strategy that leverages a set of prior models containing the true transition kernel and reward function. By optimizing over this model set, the method derives upper and lower bounds on the Q‑function to guide exploration, providing theoretical guarantees of convergence to the optimal policy. When the model set follows a bounded‑parameter MDP structure, the optimization becomes convex, enabling finite‑time convergence under mild assumptions and demonstrating accelerated learning in simulations.

By J. S. van Hulst, W. P. M. H. Heemels, D. J. Antunes