AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,805 stories · RSS feed

arXiv AI
Aug 24

Personalized Privacy Control in LLMs via Attention Head Intervention

The paper introduces the concept of personalized privacy for large language models, allowing user‑specific disclosure preferences to guide information sharing. It presents P3Bench, a benchmark that extends contextual privacy policies with personalized rules, and shows that existing prompt‑based methods often ignore these policies. To improve compliance, the authors propose “Repair”, an inference‑time attention head intervention that aligns model responses with user‑specific privacy rules.

By Junseok Kim, Nakyeong Yang, Kyomin Jung
arXiv AI
Aug 24

Automated Trajectory Evaluation for Mobile Agents via Step-Level Consequence Reasoning and Aggregation

The paper introduces CRATE, a two‑stage vision‑language model framework that evaluates mobile agents by reasoning about each step’s consequences and aggregating this evidence to assess task completion. It also presents CRATE‑S, an extension that evaluates operational safety. Experiments show CRATE and CRATE‑S outperform existing benchmarks, achieving high F1‑scores on AndroidWorld and MobileRisk datasets.

By Pengshuai Yang, Zijing Gao, Xue Yu, Benhui Zhuang, Bo Yuan, Junlan Feng
arXiv AI
Aug 24

Why2Speak: Faithful Reasoning for Abstaining Action Policies

The paper investigates how agentic systems decide between acting and abstaining, focusing on the fidelity of their reasoning explanations. Using Qwen3‑8B in a multi‑party conversation setting, the authors compare direct decision policies, reasoning policies, supervised fine‑tuning, and reinforcement learning, finding a trade‑off: strong direct policies yield higher performance but no traceable reasoning, while reasoning policies provide an audit trail at the cost of lower recall. The study also uncovers that exposing reasoning can alter the agent’s policy and that common faithfulness metrics may overstate the alignment between reasoning and decisions.

By Shreya Mendi, Brinnae Bent
arXiv AI
Aug 24

Dual-Cache Latent Space Communication between Heterogeneous Language Models

The paper introduces XKV, a latent protocol that enables efficient communication between heterogeneous language models by translating a sharer's key‑value cache into a receiver's context. XKV overcomes limitations of prior methods by jointly pooling both caches, reconciling differing layer depths, and allowing each receiver position to retrieve its own residual in native KV geometry. Across 45 dataset‑model pairings, XKV outperforms previous protocols and text communication while using fewer parameters and achieving faster translation times.

By Jiyao Liu, Qi Zhang, Yaoyi Jia, Ziwen Kan, Song Wang
arXiv Machine Learning
Aug 24

BIPPO: Budget-Aware Independent PPO for Energy-Efficient Federated Learning Services

BIPPO (Budget-aware Independent Proximal Policy Optimization) is a multi‑agent reinforcement learning framework designed for energy‑efficient client selection in federated learning (FL) over IoT systems. It addresses infrastructure constraints such as limited resources and device churn, which traditional FL and RL approaches overlook. Evaluated on two image‑classification tasks with non‑IID data, BIPPO improves mean accuracy over non‑RL methods, standard PPO, and IPPO while consuming only a negligible portion of the budget, even as client numbers grow.

By Anna Lackinger, Andrea Morichetta, Pantelis A. Frangoudis, Schahram Dustdar
arXiv AI
Aug 24

A Survey on Foundations and Frontiers of Multimodal Agentic Frameworks: Techniques and Applications

The paper surveys how large multimodal models (LMMs) enhance agentic frameworks that combine perception, memory, reasoning, planning, and action. It examines the integration of multiple modalities—text, images, audio, and video—through delegated, late‑fusion, and early‑fusion architectures, and maps these designs to agent capabilities. The survey also reviews multimodal agentic systems in robotics, web navigation, multimedia content creation, and video understanding, evaluating performance, efficiency, and scalability trade‑offs.

By Neel Mokaria, Rishie Raj, Dheeraj Baiju, Xiaoqian Shen, Shraman Pramanick, Kevin Qinghong Lin, Arda Senocak, Mike Zheng Shou, Philip Torr, Mohamed Elhoseiny, Yapeng Tian, Ruohan Gao, Salman Khan, Sayan Nag, Sanjoy Chowdhury, Dinesh Manocha
arXiv AI
Aug 24

ProofJudge: Tool-Grounded LLM Evaluation of Formal Proof Quality in Mathlib

ProofJudge is an agentic large language model that evaluates the quality of formal proofs in Lean 4 beyond mere correctness. It scores proofs on five dimensions—library leverage, automation fit, structural clarity, statement quality, and Mathlib conventions—using tool access to the relevant repository commit. The system was tested on 218 Mathlib declarations, achieving alignment with human reviewers between 63.5% and 80.8% and releasing its harness, dataset, and traces for open research.

By Shane Caldwell
arXiv AI
Aug 24

ExpertIVS: Sociological Expert Driven Individual Value Simulation in Large Language Models

ExpertIVS is a framework that uses 14 sociological expert agents to interpret World Values Survey responses, reconstructing individual value systems in a coherent, internally consistent manner rather than simply concatenating survey answers. It introduces a multi‑agent debate mechanism to assess LLM alignment with these value profiles during dynamic interactions. Experiments on 480 individuals from 12 countries show a 90.78% value restoration fidelity and a 5.3% improvement in value generalization over baseline methods, while also demonstrating strong personality discriminability and behavioral consistency.

By Zhen Wang, Yuqi Ren, Yuehan Cui, Hongxiang Wang, Jianxiang Peng, Zhaoxia Zhang, Bingkun Zhu, Tongxuan Zhang, Dezhi Tong, Deyi Xiong
arXiv AI
Aug 24

FL-MAESTRO: Multi-Agent LLM Orchestration for Resource-Constrained Federated Learning

FL-MAESTRO is a multi‑agent orchestrator that uses three specialized large language model agents to jointly decide the communication topology, per‑client resource allocation, and aggregation rule in each federated learning round. A coordinator merges the agents’ analyses, and a non‑LLM feasibility check validates the decision before execution. By filtering out clients whose updates would never be aggregated, the system eliminates the main source of wasted round energy in volatile edge networks and works across heterogeneous device classes without per‑class energy models, achieving comparable accuracy to the best energy‑aware baseline while reducing wasted energy from over a third to near zero on a non‑IID CIFAR‑10 benchmark.

By Jiajun Wu, Zirui Wang, Jiayu Zhou, Qiang Ye, Steve Drew
arXiv AI
Aug 24

TRACE: Training-time Report-guided and Clinically Ordered Concept Editing

TRACE is a training-time framework that uses structured radiology reports to guide concept editing, allowing image-only diagnosis during inference. It refines image-derived concepts with a teacher-guided editing mechanism in a malignancy-aware ordered concept space and introduces Strategic Concept Missing Training to handle incomplete annotations. The authors also present BUSC, a benchmark linking images, labels, and structured attributes, and show that TRACE outperforms existing methods on multiple datasets with better cross-domain robustness.

By Wentao Yue, Tianyou Lai, Jiayu Luo, Qingyu Mao, Ziying Wang, Zhenyuan Ning, Qilei Li
arXiv AI
Aug 24

WA-JEPA: Rethinking the Video JEPA Paradigm for World-Action Modeling in Autonomous Driving

arXiv:2608.20974v1 Announce Type: cross Abstract: Video Joint Embedding Predictive Architecture (V-JEPA) learns powerful spatiotemporal representations from video through self-supervised latent featu...

By Xinlin Wang, Yujiao Xiang, Yuheng Zhou, Jingqi Wang, Minqing Huang, Jiajie Huang, Dongxu Wei, Tingguang Zhou, Xiyang Wang, Gong Chen, Zhi Xu, Feiyang Tan, Hangning Zhou, Mu Yang