arXiv AI

DRIP-R: A Benchmark for Decision-Making and Reasoning Under Real-World Policy Ambiguity in the Retail Domain

arXiv:2605. 07699v2 Announce Type: replace-cross Abstract: LLM-based agents are increasingly deployed for routine but consequential tasks in real-world domains, where their behavior is governed by inherently ambiguous domain policies that admit multiple valid interpretations.

arXiv AI

Jun 16

RetailBench: Benchmarking long horizon reasoning and coherent decision making of LLM agents in realistic retail environments

arXiv:2606. 15862v1 Announce Type: new Abstract: Large language model (LLM) agents have made rapid progress on short-horizon, well-scoped tasks, yet their ability to sustain coherent decisions in dynamic long-horizon environments remains uncertain.

By Linghua Zhang, Jun Wang, Jingtong Wu, Zhisong Zhang

llms agents benchmarks

arXiv AI

Jul 10

RetailBench: Evaluating Long-Horizon Autonomous Decision-Making and Strategy Stability of LLM Agents in Realistic Retail Environments

arXiv:2603. 16453v3 Announce Type: replace Abstract: Large language model (LLM) agents have made rapid progress on short-horizon, well-scoped tasks, yet their ability to sustain coherent decisions in dynamic long-horizon environments remains uncertain.

By Linghua Zhang, Jun Wang, Jingtong Wu, Zhisong Zhang

llms agents benchmarks

arXiv AI

Aug 3

What Makes a Sale? Simulating End-to-End Seller--Buyer Retail Dynamics with LLM Agents

arXiv:2604. 04468v2 Announce Type: replace Abstract: Evaluating retail strategies before deployment is difficult, as outcomes are determined across multiple stages, from seller-side persuasion through buyer-seller interaction to purchase decisions.

By Jeonghwan Choi, Jibin Hwang, Gyeonghun Sun, Minjeong Ban, Taewon Yun, Hyeonjae Cheon, Hwanjun Song

llms agents

arXiv AI

Jun 12

Iterating Toward Better Search: A Two-Agent Simulation Framework for Evaluating Agentic Search Architectures in E-Commerce

arXiv:2606. 12924v1 Announce Type: new Abstract: We present a modular two-agent simulation framework for evaluating conversational shopping assistant architectures.

By Jetlir Duraj, Jayanth Yetukuri, Shuang Zhou, Dhruv Varma, Rui Kong, Ishita Khan, Qunzhi Zhou

llms agents

arXiv AI

Jun 10

T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains

arXiv:2606. 11070v1 Announce Type: cross Abstract: Recent advances in reasoning and tool-calling capabilities of large language models (LLMs) have enabled increasingly capable agentic systems.

By Genta Indra Winata, Amartya Chakraborty, Yuzhen Lin, Swasthi P Rao, Shikhhar Siingh, Houhan Lu, Nadia Bathaee, Sriharsha Hatwar, Paresh Dashore, Anmol Jain, Kshitij Tayal, Xiuzhu Lin, Anirban Das, Sambit Sahu, Shi-Xiong Zhang

llms agents benchmarks

arXiv AI

Jul 9

Large Behavior Model: A Promptable Digital Twin of the Retail Customer

arXiv:2607. 06993v1 Announce Type: new Abstract: Customer behavior modeling underpins recommendation, marketing, and decision support, yet existing approaches either optimize predictive accuracy without explaining decisions or simulate users without grounding them in real behavioral data.

By Wachiravit Modecrua, Krittin Pachtrachai, Touchapon Kraisingkorn

llms rag reinforcement-learning fine-tuning benchmarks

arXiv AI

Jul 15

Operationalising Multi-Dimensional Evaluation for Conversational Agents: A Scalable, Governed Pipeline with Selective Re-evaluation and Model Benchmarking

arXiv:2607. 12085v1 Announce Type: new Abstract: Evaluating retail conversational agents requires methods beyond lexical-overlap metrics to assess intent alignment, factuality, helpfulness, clarity, tone, and overall response quality.

By Niranjan Kumar M, Balaji Nagarajan, Karthik Nair, Faysal Satter, Nithin Surendran

llms agents benchmarks safety

arXiv AI

Jun 19

LedgerAgent: Structured State for Policy-Adherent Tool-Calling Agents

arXiv:2606. 20529v1 Announce Type: new Abstract: Policy-adherent tool-calling agents in customer-service domains must maintain task states across turns while calling tools and obeying domain policies.

By Md Nayem Uddin, Amir Saeidi, Eduardo Blanco, Chitta Baral

agents

arXiv AI

Aug 11

ComboShoppingBench: Evaluating LLM Agents for Budget-Constrained Basket Shopping with Coupons

arXiv:2608. 09282v1 Announce Type: new Abstract: Real-world shopping often requires constructing a basket of complementary items rather than retrieving a single product.

By Adrian Li, Kelong Mao, Yudong Guo, Heming Xia, Xinwei Yang, Lirui Luo, Jace Wong, Pu Yao, Sulong Xu, Simiu Gu

llms agents benchmarks

arXiv AI

Jun 11

From Explicit Elements to Implicit Intent: A Predefined Library for Auditable Behavioral Inference

arXiv:2606. 11207v1 Announce Type: new Abstract: We present SemantiClean, a modular framework for extracting structured semantic signals from e-commerce session data and driving pluggable inference targets including purchase intent, customer segmentation, and product affinity through a shared element library.

By Liu hung ming

llms computer-vision safety

arXiv AI

Jun 10

$\tau$-Rec: A Verifiable Benchmark for Agentic Recommender Systems

arXiv:2606. 10156v1 Announce Type: cross Abstract: As recommender systems transition toward agentic, multi-turn conversational interfaces, evaluation paradigms have struggled to keep pace.

By Bharath Sivaram Narasimhan, Karthik R Narasimhan

llms agents benchmarks

arXiv AI

Jul 10

SimRPD: Optimizing Recruitment Proactive Dialogue Agents through Simulator-Based Data Evaluation and Selection

arXiv:2601. 02871v3 Announce Type: replace Abstract: Task-oriented proactive dialogue agents play a pivotal role in recruitment, particularly for steering conversations towards specific business outcomes, such as acquiring social-media contacts for private-channel conversion.

By Zhiyong Cao, Dunqiang Liu, Qi Dai, Haojun Xu, Huai Yuen Khor, Hao Wang, Huan He, Yafei Liu, Ke Ma, Ruqian Shi, Sicheng Zhou, Sijia Yao

agents reinforcement-learning fine-tuning

Related stories