arXiv:2605.16116v2 Announce Type: replace
Abstract: Developing and evaluating e-commerce web agents requires environments that preserve meaningful task structure while enabling controllable, reproduc...
By Chinmay Savadikar, Mingyu Zhao, Yuanzheng Zhu, Han Li, Shuang Xie, Alberto Castelo, Tianfu Wu, Lingyun Wang
arXiv:2606. 18191v1 Announce Type: new Abstract: Deep research (DR) systems are increasingly used for complex information-seeking tasks, but existing works mainly focus on generating reports and summaries.
By Md Tawkat Islam Khondaker, Raymond Li, Muhammad Abdul-Mageed, Laks V. S. Lakshmanan, Issam H. Laradji
arXiv:2604. 04468v2 Announce Type: replace Abstract: Evaluating retail strategies before deployment is difficult, as outcomes are determined across multiple stages, from seller-side persuasion through buyer-seller interaction to purchase decisions.
By Jeonghwan Choi, Jibin Hwang, Gyeonghun Sun, Minjeong Ban, Taewon Yun, Hyeonjae Cheon, Hwanjun Song
arXiv:2606. 31693v1 Announce Type: cross Abstract: The wave of AI-native applications is moving shopping beyond page- and feed-based browsing toward intent-driven experiences orchestrated by LLM agents.
By Jiacheng Chen, Tao Zhang, Manxi Lin, Dunxian Huang, Teng Shi, Honghao Fu, Mengyan Li, Xinming Zhang, Chenchi Zhang, Xuan Lu, Xiaoxiong Du, Haibin Chen, Shaolin Ye, Hao Chang, Xiaoqi Li, Shuwen Xiao, Yujin Yuan, Jingxuan Feng, Shaopan Xiong, Huimin Yi, Ju Huang, Qiu Shen, Ying Chen, Junjun Zheng, Xiangheng Kong, Yuning Jiang
AgentOrch is a benchmark that compares global orchestration—where a complete workflow is generated upfront—with step‑by‑step interaction on 650 enterprise standard operating procedures and 3,220 runtime variants. Across all evaluated models, step‑by‑step remains more accurate, but the performance gap has narrowed by about 79% from early to recent models. The study identifies three methods—incremental workflow construction, revising workflows with execution feedback, and targeted training on complete workflows—that can further close the gap, improving global orchestration accuracy by up to 12.18 percentage points.
By Zijie Liu, Qi Yang Huang, Jinhao Duan, Yueh Che Lee, Thomas A. Joseph, Zayne Turner, Adam Seligman, Tianlong Chen, Zhaozhuo Xu
arXiv:2610.14428v1 Announce Type: new
Abstract: Enterprise workflow automation typically requires retrieving a complete and accurate set of relevant information from large operational document corpor...
By Istok Bako, Bonan Min, Tiffany Shao, Claire Cheng, Yashar Mehdad
arXiv:2609.13561v1 Announce Type: new
Abstract: Efficient utilization of supply chain analytics for decision making remains a significant challenge for planners, as critical tasks such as database qu...
By Xian Yeow Lee, Teppei Inoue, Haiyan Wang, Chetan Gupta
arXiv:2606. 15862v1 Announce Type: new Abstract: Large language model (LLM) agents have made rapid progress on short-horizon, well-scoped tasks, yet their ability to sustain coherent decisions in dynamic long-horizon environments remains uncertain.
By Linghua Zhang, Jun Wang, Jingtong Wu, Zhisong Zhang
PACEShop introduces a new evaluation framework, PACE, for shopping assistants that emphasizes personalized, actionable, compositional, and evidence‑grounded responses. The benchmark dataset contains 22,625 records with structured personas, auditable evidence pools, and detailed defect annotations, while PACEJudge offers a training‑free protocol for assessing these dimensions. Experiments demonstrate that generic judges miss key diagnostic fields, whereas PACEJudge improves evaluation across persona alignment, cross‑component consistency, grounding, and defect localization without retraining.
By Weimin Lyu, Chen Luo, Guangrui Li, Yaochen Xie, Dhineshkumar Ramasubbu, Arief Koesdwiady, Wanqiu Long, Hansu Gu, Yutong Chen, Zheshen Wang, Dakuo Wang, Yi Liu
UniACE is a unified framework that standardizes the evaluation of large language model (LLM) agents by representing each benchmark as an instruction–tool–environment triplet and running models through a shared, task‑agnostic harness in isolated runtimes. It preserves native success criteria, offers an offline mode for dynamic‑resource tasks, and standardizes efficiency metrics, execution records, and failure attribution. Applying UniACE to 7 benchmarks across 24 domains and 15 models revealed significant score shifts, ranking reversals, and sensitivity to evidence representation, highlighting the impact of evaluation configuration on reported agent performance.
By Pengyu Zhu, Lijun Li, Yaxing Lyu, Qianxin Luo, Jingyi Yang, Yi Liu, Tingfeng Hui, Xinyu Yuan, Li Sun, Sen Su, Jing Shao
arXiv:2608. 11022v1 Announce Type: cross Abstract: Model Cards and Data Cards have demonstrated the value of structured, human-readable documentation for machine learning artifacts, capturing their context, parameters, limitations, and intended use.
By Nicola Giuseppe Marchioro, Gabriele Padovani, Amal Gueroudji, Rafael Ferreira da Silva, Wesley Brewer, Valentine Anantharaj, Sandro Fiore, Renan Souza
arXiv:2609.06128v1 Announce Type: new
Abstract: Production LLM agents execute tool-calling loops, retrieval chains, and compositional workflows in multiple modes, yet execution semantics are often co...
By Tarun Gopinath, Atul Kulkarni, Vijay Rajakumar, Shrikar Katti, Parthasarathy Govindarajen