arXiv AI

SportD: How do VLMs physically strategize?

arXiv:2607. 14616v3 Announce Type: replace Abstract: Vision-language models (VLMs) can describe a scene, but can they act well within one?

arXiv AI

Jul 17

SportD: Can VLMs Physically Strategize?

arXiv:2607. 14616v1 Announce Type: new Abstract: Vision--language models have become increasingly capable of interpreting visual scenes, but it remains unclear whether they can use information to make strategically effective decisions.

By Jasin Cekinmez, Addison J. Wu, Haotian Xia, Akshaya Bharadhwaj, Anay Putty, Anirudh Ravishankar, Jaewoong Lee, Jinglin Xiao, Kyumin Andrew Shim, Mishika Ahuja, Nisarga Patil, Leo Liu, Zhuohan Liu, Weining Shen

llms benchmarks

arXiv Machine Learning

Jun 16

How Should World Models Be Evaluated? A Decision-Making-Centric Position

arXiv:2606. 15032v1 Announce Type: new Abstract: World models have rapidly become one of the central abstractions in modern AI.

By Yang Yu, Shiyuan Zhang, Yifei Sheng, Haoxiang Ren, Haoxin Lin

robotics benchmarks

arXiv Machine Learning

Aug 4

Humans Are More Diverse: Frontier LLMs Show Extreme Policies in Idealised AI Development Races

arXiv:2608. 01193v1 Announce Type: cross Abstract: An AI development race creates a multi-agent safety dilemma.

By Phu Hoa Pham, Duy Minh Dao Sy, Trung Kiet Huynh, Phu Quy Nguyen Lam, Chi Nguyen Tran, Minh Trung Le, Phong Hao Le, Dinh Nam Nguyen, Thien Ky Nguyen Dong, Elias Fernandez Domingos, Le Hong Trang, The Anh Han

llms agents benchmarks safety

arXiv AI

Jul 21

FIFA World Cup 2026 as a Contamination-Free Benchmark for LLM Forecasting Agents: Four Models, a Bookmaker, and 104 Matches

arXiv:2607. 17765v1 Announce Type: cross Abstract: We introduce WC2026-Agents, a benchmark and dataset for evaluating large language models (LLMs) as autonomous forecasting agents on real, future events.

By Jiacheng Ding, Cong Guo, Jason Xu

llms agents benchmarks

arXiv AI

Jul 1

Position: Vision-Language-Action Models Cannot Be Verified to Perform Physical Reasoning

arXiv:2606. 30686v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) systems, built on pretrained vision-language models (VLMs), have shown rapidly improving performance on robot manipulation benchmarks.

By Taozhao Chen, Ian Manchester, Huaming Chen

llms robotics multimodal benchmarks

arXiv Machine Learning

Jun 30

How Should World Models Be Evaluated for Embodied Decision-Making? A Decision-Making-Centric Position

arXiv:2606. 15032v2 Announce Type: replace Abstract: World models have become a central abstraction in modern AI.

By Yang Yu, Shiyuan Zhang, Yifei Sheng, Haoxiang Ren, Haoxin Lin

robotics

arXiv AI

Jun 18

RTSGameBench: An RTS Benchmark for Strategic Reasoning by Vision-Language Models

arXiv:2606. 18950v1 Announce Type: new Abstract: Modern Vision-Language Models (VLMs) often struggle with strategic reasoning, i.

By San Kim, Daechul Ahn, Reokyoung Kim, Hyeonbeom Choi, Seungyeon Jwa, Jonghyun Choi

llms agents multimodal benchmarks

arXiv AI

Jul 7

Out-of-Distribution Generalization of Risk Aversion in Language Models

arXiv:2607. 02755v1 Announce Type: cross Abstract: Training AIs to be risk-averse in resources could offer a failsafe in the event that AIs turn out misaligned.

By Kristina Zhang, Junior Chinomso Okoroafor, Benjamin Maltbie, Andrew Lin, Abhitej Bokka, Elliott Thornley

llms reinforcement-learning fine-tuning benchmarks

arXiv AI

Aug 3

DungeonBench: A Benchmark for Rules-Rich Tactical Reasoning in Dungeons & Dragons Combat

arXiv:2607. 29577v1 Announce Type: new Abstract: Games and simulators make valuable benchmarks by turning decisions into measurable outcomes, but many current suites under-test rules-rich tactical reasoning: the ability to choose well when geometry, timing, resources, objectives, and rule interactions all matter at once.

By Ismayil Ismayilov, Atakan Kara, Kaan Oktay

agents benchmarks

arXiv AI

Jun 2

VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Models

arXiv:2601. 03309v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models, which integrate pretrained large Vision-Language Models (VLM) into their policy backbone, are gaining significant attention for their promising generalization capabilities.

By Jianke Zhang, Xiaoyu Chen, Qiuyue Wang, Mingsheng Li, Yanjiang Guo, Yucheng Hu, Jiajun Zhang, Shuai Bai, Junyang Lin, Jianyu Chen

llms computer-vision robotics fine-tuning multimodal benchmarks

arXiv AI

Jun 10

Superficial Beliefs in LLM Decision-Making

arXiv:2606. 11016v1 Announce Type: new Abstract: We ask whether large language models (LLMs) merely imitate rationales when choosing between two options, or whether their choices reflect a systematic underlying decision structure.

By Gabriel Freedman, Francesca Toni

llms

arXiv AI

Jun 26

Automating Potential-based Reward Shaping with Vision Language Model Guidance

arXiv:2606. 27180v1 Announce Type: cross Abstract: Sparse rewards are inherently challenging for reinforcement learning agents as they lack intermediate feedback to guide exploration and to correctly attribute the sparse success rewards to relevant parts of the trajectory.

By Henrik M\"uller, Daniel Kudenko

llms agents reinforcement-learning multimodal

Related stories