Large language model (LLM) agents now fix software, operate computers and serve customers, but their harnesses, the programs that run them, call a large LLM at every step, which makes long tasks slow...
arXiv:2605. 16301v2 Announce Type: replace-cross Abstract: Evaluating animal welfare reasoning in LLMs remains an open challenge despite rapid deployment in consumer and professional contexts where welfare considerations appear implicitly in everyday queries.
By Isabella Luong, Joyee Chen, Arturs Kanepajs, Jasmine Brazilek, Sankalpa Ghose, David Williams-King, Linh Le, Allen Lu
JevAgentHarness is a new framework that benchmarks, deploys, and trains decision models to share control with large language models (LLMs) in agentic tasks. It introduces AgentDecisionBench to evaluate decision models from isolated questions to full agent control, revealing that accurate isolated decisions still miss many tasks and that naïve delegation can reduce success. By fitting confidence thresholds per choice and training a 4B decision model on states after agent mistakes, JevAgentHarness achieves high task success while cutting cost and speeding up execution, with results and resources made publicly available.
By Kunlun Zhu, Tao Feng, Xu Han, Heng Wang, Ling Yang, Xiangru Tang, Jiaxuan You, Heng Ji
CrashGym is a deterministic, replayable limit‑order‑book simulator that lets language‑model agents decide between a deleveraging intervention and a liquidity backstop before a market crash. Agents can query intervention experiments in a laboratory version of the market, and their choices are scored by simulating every feasible intervention on an independently seeded target realization, allowing direct regret calculation. In a study of 24 new worlds, laboratory access reduced mean regret by 57.4% for one deployment, and exploratory results suggest lower regret across all evaluated deployments, while an evolving strategy mix offers no clear advantage over a fixed mix.
By Haocheng Lu, Haoxuan Cheng, Mingyang Chen, Minjun Zhu
Language-model agents are increasingly asked to run experiments, yet benchmarks rarely measure whether those experiments improve the decisions they are meant to inform. We introduce CrashGym, a determ...
The paper introduces a continued‑pretraining corpus called "coupled welfare" that embeds ecological reasoning into large language models. After a single QLoRA run, the corpus was applied to five open models across three architectures, resulting in a dramatic reduction of harmful decisions under high‑pressure scenarios while leaving general performance unchanged. The study also demonstrates that the improvement is content‑specific, as a neutral control corpus did not yield similar gains, and that the models remain robust against adversarial attempts to justify environmentally destructive actions.
By Trent R. Northen, Mingxun Wang