The paper investigates the problem of sharing a single critic across multiple parallel environments in reinforcement learning. It shows that when environments assign different expected returns to the same state, a shared critic must reconcile conflicting value targets, which can distort advantage estimates and misguide policy updates. The authors propose a simple fix—providing the critic with the environment index—demonstrating through bandit models and experiments on CartPole, MuJoCo, BipedalWalker, and 16 Procgen games that this conditional critic stabilizes learning and boosts returns, achieving a 40.8% improvement in aggregate normalized return on unseen levels.
By Zhenya Liu, Yang Meng, Zhuokai Zhao, Xuefeng Liu, Yuxin Chen
arXiv:2605. 13909v2 Announce Type: replace-cross Abstract: Negotiation is a central mechanism of economic exchange, shaping markets, procurement, labor agreements, and resource allocation.
By Erica Zhang, Fangzhao Zhang, Aneesh Pappu, Batu El, Jose Blanchet, Susan Athey, Jiashuo Liu, James Zou
arXiv:2607. 04470v1 Announce Type: cross Abstract: Large Language Models (LLMs) offer a natural interface for translating human objectives into reward signals for cooperative multi-agent reinforcement learning (MARL), yet the training-time dynamics of this integration remain poorly understood.
By Faid Keddouri, Sohaib Houhou, Aissa Boulmerka, Nadir Farhi
arXiv:2602. 12963v2 Announce Type: replace Abstract: An important question in the field of AI is the extent to which successful behaviour requires an internal representation of the world.
By Alfred Harwood, Jose Faustino, Alex Altair
arXiv:2606. 27397v1 Announce Type: cross Abstract: Evaluating LLM agents requires dynamic environments that go beyond static reasoning and zero-sum games.
By Yeqi Feng, Yuxin Chen, Tianxing He
arXiv:2606. 27032v1 Announce Type: cross Abstract: Energy trading decisions depend not only on current market prices, but also on expected future market conditions, and operational constraints.
By Jesper Klicks, Sander Vr\v{z}ina, Vincent Fran\c{c}ois-Lavet
arXiv:2608. 01425v1 Announce Type: cross Abstract: Training LLM-based multi-agent systems with multi-agent reinforcement learning is rapidly gaining traction, and a parallel line of work argues that such systems should be judged by their behavior, not only their reward.
By Yi Mao, Andrew Perrault
arXiv:2608. 03076v1 Announce Type: new Abstract: Multi-agent studies commonly place AI agents in predefined games, markets, or roles, making it difficult to distinguish endogenous economic organization from behavior inherited from the scenario.
By Lingyun Zhang, Shang Shang
arXiv:2605. 25739v2 Announce Type: replace Abstract: We prove that no reinforcement learning policy with confidence-gated autonomy can simultaneously achieve maximum helpfulness, optimal calibration, and full autonomy under rational oversight, whenever some tasks exceed the agent's reliable competence: the Behavioral Credibility Trilemma.
By Lauri Lov\'en, Nam Do, Hassan Mehmood, Dinesh Kumar Sah, Sasu Tarkoma
arXiv:2608. 01208v1 Announce Type: cross Abstract: For a trading desk, residual climate hedging valuation adjustment (HVA) is the climate cost left after its inherited hedge and any admissible overlay have been taken into account; it therefore cannot be inferred from a stand-alone stress loss.
By Xiaozhen Wang, Francois Buet-Golfouse
The paper investigates how a single pooled contract offered by an aggregator to heterogeneous smallholder farmers can be designed to maximize profit while addressing private adoption costs and unobserved effort over multiple seasons. Using a POMDP framework and reinforcement learning, the authors find that profit‑maximizing contracts disproportionately favor large farms, achieving 87.7% of possible adoption on large farms versus only 8.2% on smallholdings, largely due to higher measurement, reporting, and verification costs on smaller plots. The study suggests that adjusting MRV cost structures could reduce this disparity and help scale carbon farming to smallholders.
By Rishi Bharadwaj, Yadati Narahari
arXiv:2608. 02713v1 Announce Type: cross Abstract: Continually improving agents require dynamic interaction feedback beyond static supervision, yet direct real-environment interaction is costly, slow, unsafe, and hard to parallelize.
By Yu Yang, Xuemeng Yang, Licheng Wen, Lingdong Kong, Xiaobin Hu, Dongyue Lu, Wei Chow, Xiyan Huang, Yuxiang Feng, Yue Liao, Jianbiao Mei, Daocheng Fu, Rong Wu, Pinlong Cai, Ran Yi, Ying Tai, Jiangning Zhang, Botian Shi, Yong Liu, Shuicheng Yan