Large language models access knowledge inconsistently across languages, but to what extent do they differ in their skill sets when interacting with different languages? This work quantifies cross-ling...
The paper investigates whether large language models (LLMs) exhibit language‑specific skill differences by having two identical model instances play a text‑based game in different languages. Using a multilingual extension of TextArena, the authors evaluate three open‑weight models across eight languages and six games, finding that the same model can show markedly different performance—varying win–loss margins, invalid actions, and strategic choices—depending on the language interface. Analyses pinpoint language‑specific failures in spatial reasoning, card‑conditioned decisions, and optimal move selection, and demonstrate that adjusting the intermediate reasoning language can recover much of the lost performance.
arXiv:2607. 27536v1 Announce Type: cross Abstract: Learning a strategic task changes more than what is directly taught: fine-tuning on one game can either enhance or degrade an agent's ability to reason in another.
arXiv:2404. 02039v5 Announce Type: replace Abstract: Game environments provide rich, controllable settings that stimulate many aspects of real-world complexity.
arXiv:2609.00474v1 Announce Type: cross Abstract: LLMs are increasingly deployed as orchestrators that coordinate specialized subagents to solve complex tasks through natural language. However, in ma...
The paper presents a post‑training recipe for small dialogue‑game agents that involves three steps: acquiring broad game participation via supervised fine‑tuning, repairing specific local failures with turn‑local preference pairs, and preserving general capabilities. Applied to the LM Playschool Challenge, the method raises the public clemscore from 10.67 to 38.92 and the closed in‑domain score from 13.41 to 41.17 while keeping overall static performance nearly unchanged. The gains are mainly within the targeted game family, with limited improvement on out‑of‑domain clemscore.
The paper presents the first dataset that captures how humans learn and use mechanisms in unfamiliar environments, using ARC‑AGI‑3 as a testbed. It includes 144 trajectories from 64 participants across 25 public games, each paired with time‑synchronized actions and think‑aloud audio, and 250 trajectories from large language model agents with and without harnesses. The dataset is annotated for mechanism triggers, action purposes, errors, and recovery, enabling a comparison of human and LLM cognition, highlighting differences in hypothesis testing, exploration, and use of qualitative representations.
arXiv:2608. 20274v1 Announce Type: new Abstract: Large language model (LLM) agents can induce skills from completed tasks and reuse them later to grow more capable with experience.
Learn2Play Bench is a new benchmark that tests how well large language model agents learn from experience in unfamiliar, text‑based games with novel or counterintuitive rules. The benchmark provides reproducible feedback, automatic scoring, and varied game instances to evaluate learning across repeated attempts and transfer to new situations. Findings show that retaining full action records aids learning, human players outperform agents, and the choice of harness significantly impacts performance and inference cost.
The paper introduces Qwen-GuidePlay-2B, a 2B‑parameter language model fine‑tuned for dialogue‑game interaction. The training pipeline consists of three stages: supervised fine‑tuning on successful game trajectories, weighted turn‑level fine‑tuning, and teacher‑guided fine‑tuning that corrects formatting and evaluates examples. The resulting model achieves a clemscore of 57.12 and a statscore of 42.68 on Playpen’s public validation, ranking second in the official challenge and demonstrating that careful curation can outperform more aggressive procedural methods.
arXiv:2509. 26306v5 Announce Type: replace Abstract: Existing multi-agent learning approaches have developed interactive training environments to explicitly promote collaboration among multiple Large Language Models (LLMs), thereby constructing stronger multi-agent systems (MAS).
arXiv:2607. 00527v1 Announce Type: new Abstract: Generative AI now enables games to produce dialogue, quests, characters, images, and worlds at runtime.