RealWorldShop introduces a new benchmark for conversational shopping agents, featuring 3.28 million grounded products, structured shopping episodes, a profile‑grounded user simulator, and role‑play evaluation. Analysis reveals that existing systems generate locally plausible responses but struggle with state tracking, constraint updating, and grounded convergence, especially in ambiguous or multi‑intent scenarios. The authors propose REALSHOP_AGENT, a session‑control framework with explicit state management, shopping‑flow control, catalog‑grounded retrieval, and runtime guards, which consistently outperforms strong baselines on the benchmark.
By Xinwei Yang, Kelong Mao, Yudong Guo, Sulong Xu, Simiu Gu, Chen Huang, Wenqiang Lei
The study reports a $2 imes2$ randomized field experiment involving 1,072 calls to a German Santa Claus telephone hotline, with 89 child conversations (median age 6) meeting inclusion criteria. Children were routed to one of four LLM voice agents that varied in persona (Santa vs. Helper) and framing (persuasive nudges toward prosocial wishes vs. neutral). Persuasive framing increased the likelihood of a prosocial wish from 11.6% to 45.7%, while persona authority had little effect on compliance but did influence engagement, with children more likely to hang up on the Helper within the first minute.
By Thilo Tamme (Technical University of Munich), David Steck (Technical University of Munich), Anton Hantel (Massachusetts Institute of Technology)
OSWorld-Science is a benchmark and evaluation environment for computer-using agents that use visual language models (VLMs) to perform scientific software tasks. It includes 12 VLMs and 146 high-quality tasks across domains such as molecular drawing, pathology image analysis, statistical computing, and physical simulation, with artifact-based evaluation and a harness that logs interactions and supports model comparison. The benchmark was developed through expert proposals and iterative human–AI co‑design, and results show that current VLMs still struggle with key scientific questions, offering insights into factors like language, reasoning, and context length.
By Dingyuan Dai, Heli Qi, Lei Liu, Yinxi Li, Baiding Chen, Zijun Dou, Qingcheng Zeng, Qi Kang, Oliver Sun, Eric Wang, Bo Zhou, Haixin Wang, Yufan Du, Shi Bo, Ruihan Lin, Mengqi Yuan, Dunjie Lu, Steven Dillmann, Yiming Shi, Tina Su, Amy Xin, Minghao Liu, Xi Wang, Xu Huang, Ge Zhang, Pengyu Nie, Zhen Yang, Jie Tang, Juanzi Li, Weihao Xuan, Tianyu Liu
The article reviews how replacing components in language‑model agents affects execution trajectories and downstream outcomes. It maps 348 studies, analyzes 90 comparison records, and finds that while many studies report both local decision metrics and task endpoints, they rarely demonstrate matched comparisons or prove that improved local decisions drive task‑level gains. The review identifies three potential mechanisms—recovery and disruption, intervention timing, and downstream use—and proposes eight claim‑specific reporting items to clarify evidence quality.
By Shuyang Zhang (The Hong Kong Polytechnic University), Jianshuo Chang (The Hong Kong Polytechnic University)
E2E-SWE is a benchmark that tests large language models’ ability to create complete, functional software repositories from scratch. It includes 186 tasks across 11 programming languages, each requiring an agent to build an installable project based solely on a natural‑language specification and an empty workspace, while passing a hidden test suite. The benchmark was crafted by software engineers and LLMs, then refined through iterative verification by autonomous agents to ensure clarity and solvability.
By Hantian Ding, Chloe Bi, Jiacheng Zhu, John Yang, Matt Deitke, Pengcheng Yin, Zijian Wang, Rui Hou
The paper introduces a reusable Semantic Web framework that aggregates fragmented evidence needed for Fundamental Rights Impact Assessments under the EU AI Act, focusing on high‑risk public sector categories such as employment and worker management and access to essential public services. A curated 150‑record corpus is annotated across four axes and serialized into a SPARQL‑queryable knowledge graph of 1,351 RDF triples, enabling five demonstration scenarios that retrieve 103 records (68.7% coverage). Evaluation against a 69‑record gold standard shows that LLM‑assisted classification in the employment domain yields a low κ of 0.045, highlighting challenges in automated fairness‑related evidence retrieval, while all artefacts are released openly for regulators, authorities, and SMEs.
By Faith Olopade, Delaram Golpayegani, David Lewis
The article discusses a notable imbalance in AI security research, where studies on attacking AI systems outnumber those on defending them. It highlights that this skew is evident across various subfields such as federated learning, speech recognition, membership inference, and large language models. The authors argue that attack papers often benefit from favorable evaluation conditions, whereas defense papers face stricter standards, resulting in a literature rich in vulnerabilities but lacking robust, deployable protections.
By Youqian Zhang
MiniRep is a reputation‑based aggregation system designed for multi‑agent debate (MAD) that remains robust even when malicious agents are present. It evaluates agents on both their current task performance and historical reputation, while preventing groups of agents with highly similar responses from dominating the final decision. Experiments on the MATH benchmark show that MiniRep consistently outperforms conventional MAD aggregation and other reputation‑based approaches across a wide range of attack scenarios.
By Jiaming Zhang, Yuwan Liu, Yue Huang, Sisi Duan
The study explores how large language models (LLMs) can influence each other’s beliefs by simulating conversations between a target LLM and an influencer LLM. It identifies two radicalization pathways—resonance, which amplifies pre‑existing beliefs, and persuasion, which introduces new beliefs—and finds that resonance consistently produces stronger radicalization effects. The research also shows that different influence tactics yield varying levels of radicalization and that resonance can spread to related beliefs, indicating interconnected belief structures within AI agents.
By Ozgur Can Seckin, Shalmoli Ghosh, Alessandro Flammini, Kristina Lerman, Maria Elizabeth Grabe, Filippo Menczer
The paper introduces the concept of stale binding, where large language models (LLMs) answer with outdated values despite having newer information in context. It presents Controlled In-Context Memory (CICM), a benchmark to track and test the use of updated information in conversations and agent logs. Experiments on open‑source models reveal an attention drift mechanism that favors old values, and the authors propose a simple attention‑redirecting intervention that largely corrects these errors without harming correct answers.
By Junyu Guo, Yuchen Fang, Shangding Gu, Costas Spanos, James Demmel, Javad Lavaei
The study investigates how humans and AI collaborate on a puzzle task, focusing on referential uncertainty—when a description could refer to multiple objects. It finds that eliciting a belief distribution over candidate pieces yields better calibration and discrimination than raw action probabilities, and that precise descriptions or well‑targeted hedges significantly reduce the acceptance of wrong placements. However, the AI rarely externalizes uncertainty, and poorly targeted hedges can be counterproductive.
By Christian Poelitz, Finale Doshi-Velez, Si\^an Lindley
The paper introduces BACKDROP, a benchmark that evaluates how well AI agents maintain their capabilities when faced with everyday hazards in dynamic environments. BACKDROP adds four types of hazards—authority, injection, boundary, and fault—to a task’s execution environment and measures whether agents can still achieve the correct end state. Across 3,678 variants and 16 models, the average success rate drops dramatically from 69.5% to 31.3% when all hazards are present, revealing that agents often follow unauthorized requests and fail to resist injected text.
By Nusrat Jahan Lia, Shubhashis Roy Dipta
arXiv:2609.38362v1 Announce Type: new
Abstract: Generative vision-language models (VLMs) such as Qwen-VL and LLaVA achieve strong zero-shot performance on tasks overlapping with their pretraining dis...
By Hung-Jen Chen, Yu-Heng Ho, Ting-Yao Huang, Po-Hsiang Hsu, Li-Yu Chen, Chun-Yi Lee, Min Sun
arXiv:2609.39112v1 Announce Type: new
Abstract: Camera traps accumulated vast, multidimensional data for wildlife monitoring, yet translating raw media archives into meaningful ecological insights re...
By Yutong Deng, Qi Song, Xi Guo, Tianming Wang, Lei Bao, Jianping Ge
arXiv:2609.39265v1 Announce Type: new
Abstract: The Segment Anything Model (SAM) achieves remarkable performance in visual segmentation. The latest SAM3 extends promptable segmentation to concept-lev...
By Ziqi Zhou, Yifan Hu, Yufei Song, Haowen Jiang, Xianlong Wang, Shengshan Hu, Dezhong Yao, Leo Yu Zhang
arXiv:2609.40317v1 Announce Type: new
Abstract: While talking head generation has advanced rapidly, generating natural listener behavior in dyadic conversations, which know when to react, how to reac...
By Zikai Liao, Yumin Suh, Yi Ouyang, Yi-Lun Lee, Yi-Hsuan Tsai, Zhaozheng Yin
arXiv:2609.39969v1 Announce Type: cross
Abstract: Physical LiDAR attacks are often evaluated using fixed primitives and manually selected parameters, despite their strong dependence on surrounding tr...
By Yiming Gao, Shaocheng Luo
arXiv:2609.38956v1 Announce Type: new
Abstract: Routing signals of modern vision transformers -- expert gates, attention-residual weights and halting scores -- often improve probes that predict wheth...
By Wenhao Liang, Lin Yue, Wei Emma Zhang, Mingyu Guo, Olaf Maennel, Weitong Chen
arXiv:2609.40325v1 Announce Type: new
Abstract: As interactive 3D worlds are increasingly used to study intelligent behavior, it becomes important to develop efficient pipelines for identifying anoma...
By Ziyan Jiang, Jingbo Yang, Jiabao Ji, Yujian Liu, Qiucheng Wu, Tommi Jaakkola, Yang Zhang, Shiyu Chang
arXiv:2609.38266v1 Announce Type: cross
Abstract: Agentic large language models (LLMs) now move money through tools, yet the record of what they did is usually a trace their own process emits beside...
By Mustafa Arslan