E-Commerce Bench is an open‑source benchmark that simulates a year‑long e‑commerce operation, requiring LLM agents to manage multiple online stores, negotiate with suppliers, optimize sales, fulfill orders, handle returns, and manage cash flow. The environment uses real product and supplier data, a calendar of promotions and shocks, and deterministic customer and negotiation models to enable reproducible evaluation. The study evaluates 18 state‑of‑the‑art models across seven metrics, finding no single model dominates, with GPT‑5.6 Sol achieving the highest year‑end assets but lagging in fraud avoidance and operational efficiency.
By Wei Fan, Xinjie Shen, Xudong Guo, Jianhong Tu, Yang Su, Yinger Zhang, Lianghao Deng, Fengyu Wang, Baohua Dong, Yangqiu Song, Dayiheng Liu
arXiv:2610.28454v1 Announce Type: new
Abstract: Agents are starting to sell to other agents. Seller agents write the listings of tools, services and products, and buyer agents read these listings whe...
By Zhenwei Tang, Di Bai, Jintao Liu, Xidong Wu, Qianfeng Wen, Xinwu Cheng
PriceBench is a diagnostic benchmark that extracts price, quality, and brand preferences from large language models (LLMs) by analyzing their hotel booking choices. Using a logit choice model, the study evaluated 28 LLMs from eight providers across 3,600 booking tasks involving 179 New York City hotels. Results show that more capable LLMs exhibit stronger, more consistent preferences, while weaker models either lock onto a single position or show near-indifference, with significant variation in price sensitivity and price/quality trade-offs across providers.
By Pavel Kireyev
arXiv:2607. 28956v1 Announce Type: new Abstract: Large language model agents are increasingly evaluated as autonomous tool users, yet most benchmarks focus on bounded tasks with immediate success criteria.
By Qiming Shi, Yulong Tao, Linbo Jin, Zhaolu Kang, Yibo Dou, Jiawen Zhu, Tianjun Pan, Shaokang Fu, Chengyu Wang, Siyue Li, Yaping Cheng, Di Weng, Chengfu Huo
The paper reports a six‑month, population‑scale measurement of autonomous language‑model trading agents operating in two production fleets: DX Terminal Pro, with 3,505 user‑funded vaults trading real ETH in Base memecoin markets, and the DXAP live alpha fleet, with 500–599 user‑created agents trading Hyperliquid perpetuals. Across roughly 7.5 million single‑model invocations and 231,638 multi‑tool turns, the study finds that operating layer design, risk sliders, and leaderboard boundaries drive behavior more than strategy text; agents are volatility‑blind in sizing, capture little upside, and show no directional edge compared to a retail benchmark. The analysis includes regression discontinuity, permutation nulls, and a 17‑rule methodology canon to validate the findings.
By T. J. Barton, Chris Constantakis, Patti Hauseman, Annie Mous, Alaska Hoffman, Brian Bergeron, Hunter Goodreau
Language-model agents are increasingly asked to run experiments, yet benchmarks rarely measure whether those experiments improve the decisions they are meant to inform. We introduce CrashGym, a determ...
The study investigates how sales language in tool registry descriptions influences AI agents’ tool selection. By creating paired listings that differ only in praise, verifiable specifications, or order, the authors found that stacked praise increased a tool’s pick rate by about 43 percentage points, while the first-listed tool was chosen roughly 72 points more often. Structured fields for numeric limits helped agents choose capable tools, but accompanying sales text could negate this benefit.
By Haochuan Kevin Wang
CrashGym is a deterministic, replayable limit‑order‑book simulator that lets language‑model agents decide between a deleveraging intervention and a liquidity backstop before a market crash. Agents can query intervention experiments in a laboratory version of the market, and their choices are scored by simulating every feasible intervention on an independently seeded target realization, allowing direct regret calculation. In a study of 24 new worlds, laboratory access reduced mean regret by 57.4% for one deployment, and exploratory results suggest lower regret across all evaluated deployments, while an evolving strategy mix offers no clear advantage over a fixed mix.
By Haocheng Lu, Haoxuan Cheng, Mingyang Chen, Minjun Zhu
The paper investigates how role assignment in large language model (LLM) recommenders influences sponsorship bias. By assigning the agent’s principal as either a traveler or a booking platform, the authors find that platform delegation reduces the penalty applied to sponsored listings and weakens consumer skepticism triggered by disclosure. The study also shows that stricter terminology and attribution to the platform widen the divergence in agent evaluations, indicating that current disclosure mandates are insufficient to protect consumers in AI-mediated commerce.
By Davood Wadi, Yu Ma
arXiv:2601. 11496v3 Announce Type: replace-cross Abstract: AI agents increasingly mediate bargaining, negotiation and persuasion for people and firms.
By Eilam Shapira, Moshe Tennenholtz, Roi Reichart
arXiv:2608. 07538v1 Announce Type: new Abstract: As LLM agents move from decision support to autonomous procurement, firms need to know whether delegated negotiators create value, divide it predictably, and avoid money-losing contracts.
By Chen Liang, Fasheng Xu
arXiv:2607. 10202v1 Announce Type: new Abstract: Cross-model comparisons read divergence in value dispositions as evidence that language models hold individuated values.
By Hong-In Won, Jinseok Jang, Hyoseop Kim