The paper introduces a deterministic, reproducible e‑commerce environment that pre‑commits customer and trajectory parameters, enabling a simulated consumer to attempt purchasing a target cart with the help of an evaluated model. The environment records every assistant action and state, allowing post‑trial evaluation of specific conversation components and applying penalties based on tool‑call accuracy. Using this setup, the authors benchmark eight open‑weight agents (20B–35B parameters) across 160 trials and 44 metrics, revealing nuanced performance issues such as under‑action, over‑purchase, unsupported product attributes, and poor search that are hidden by overall success rates.
By Nimit Shah, Haitz S\'aez de Oc\'ariz Borde
Commercial NLP treats the shopping chatbot as a recommender or a conversion tool: its job is to match a user to a catalogue entry and close a sale. We argue that the arrival of agent-native micro-payment rails (e.
arXiv:2606. 24783v1 Announce Type: cross Abstract: Commercial NLP treats the shopping chatbot as a recommender or a conversion tool: its job is to match a user to a catalogue entry and close a sale.
By Filippos Ventirozos, Matthew Shardlow
arXiv:2606. 12924v1 Announce Type: new Abstract: We present a modular two-agent simulation framework for evaluating conversational shopping assistant architectures.
By Jetlir Duraj, Jayanth Yetukuri, Shuang Zhou, Dhruv Varma, Rui Kong, Ishita Khan, Qunzhi Zhou
arXiv:2606. 14314v1 Announce Type: new Abstract: LLM agents have rapidly evolved into autonomous systems, yet a persistent information gap remains between users and agents: communication is costly, while users' identical preferences further limit information exchange.
By Xinbei Ma, Jiyang Qiu, Yao Yao, Zheng Wu, Yijie Lu, Xiangmou Qu, Jiaxin Yin, Xingyu Lou, Jun Wang, Weiwen Liu, Weinan Zhang, Zhuosheng Zhang, Hai Zhao
Dialogue systems in e-commerce scenarios often need to satisfy multiple objectives: accurately reasoning over user profiles (e. g.
Consilience is an inference‑time orchestration framework that steers and certifies communication among multi‑agent large language models in hidden‑profile settings. It summarizes each discussion turn with a compact state of uncertainty, disagreement, evidence gain, redundancy, and premature consensus, then selects a communication intervention (challenge, clarify, seek evidence, or route) and speaker. A round‑wise conformal calibration procedure guarantees that the controller’s proposed action has bounded one‑step regret with high probability, and an acceptance mechanism enforces this guarantee for the executed action. Experiments on HiddenBench‑style tasks show that Consilience improves decision accuracy and communication efficiency over fixed and unstructured protocols, sometimes outperforming a full‑information baseline.
By Abhijith Babu, Ramneet Kaur, Vishal Pramanik, Olivera Kotevska, Nathaniel D. Bastian, Susmit Jha, Sunny Raj, Yanzhao Wu, Sumit Kumar Jha, Anirban Roy
Effective social interaction requires more than fluent responses: agents must reach plans that fit the participants' needs and circumstances. Existing evaluations of social agents mainly rely on LLM j...
arXiv:2608. 14613v1 Announce Type: new Abstract: Modern LLM-agent frameworks increasingly interoperate through standards such as Anthropic's Model Context Protocol (MCP) for agent-to-tool access and Google's Agent2Agent (A2A) protocol for agent delegation and negotiation.
By Wael Albayaydh, Rui Zhao
The paper introduces TruthMarketTwin, a simulation framework that uses agent-based modeling to study large language model (LLM) agents in e‑commerce markets characterized by asymmetric information. It models bilateral trade where sellers and buyers make strategic decisions about listings, purchases, ratings, and recourse to maximize profit and utility. The study finds that LLM agents can autonomously exploit weaknesses in reputation‑based governance, but that warrant enforcement can reduce deception and alter strategic behavior.
By Shijun Lei, Quang Nguyen, Swapneel S Mehta, Zeping Li, Huichuan Fu, Xiaolong Zheng, Siki Chen, Yunji Liang, Philip Torr, Zhenfei Yin
The paper introduces JevAct, a runtime verification framework that intercepts selected actions of large language model (LLM) agents before execution. JevAct compiles domain policies into executable state checks and uses a prefill-only decision model to provide probability‑based judgments, enabling feedback‑driven correction without modifying the underlying agent. Evaluations on eleven LLM agents across Airline, Telecom, and BFCL benchmarks show that JevAct improves agent reliability, especially on Telecom and BFCL tasks.
By Iftach Shoham, Yuval David, Offer Akrabi, Erez Bilgory, Ido Levy, Sami Marreed, Sergey Zeltyn, Segev Shlomov
arXiv:2608. 06329v1 Announce Type: cross Abstract: Task-oriented conversational agents are evaluated using curated or automatically generated benchmarks, yet benchmark quality is rarely assessed.
By Noam Koren, Roy Bar-Haim, Abigail Goldsteen