Hugging Face Trending Papers

Playing Dumb: How a Chess Transformer Simulates Users Across Skill Levels

Read the original on Hugging Face Trending Papers →

The paper investigates how a chess transformer, Maia‑2, simulates players of different skill levels by conditioning on quantitative skill ratings. By fixing board positions and model weights, the authors show that the model’s internal knowledge of 172 chess concepts remains stable across skill levels, yet lower‑skill predictions can be dramatically improved by fine‑tuning only the final output layer. This demonstrates that weaker behavior in generative models may stem from how retained knowledge is applied rather than from a lack of knowledge itself.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv AI
Oct 23

D-Skill: Distilling Teacher Knowledge through Agent Skills

D‑Skill is a framework that distills the behavioral advantage of a large language‑model teacher into an interpretable natural‑language skill document, using only paired rollouts and no weight updates. The skill is constructed offline from fixed trajectories, then provided to a smaller student model at inference time. Across benchmarks such as SpreadsheetBench‑Verified, ALFWorld, and chess puzzles, D‑Skill closes 75‑95% of the teacher‑student performance gap and reduces inference cost by 5.7×–8.3× compared to the teacher.

By Shuo Yuan, Dacheng Li, Joseph E. Gonzalez
arXiv AI
Oct 7

Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?

Learn2Play Bench is a new benchmark that tests how well large language model agents learn from experience in unfamiliar, text‑based games with novel or counterintuitive rules. The benchmark provides reproducible feedback, automatic scoring, and varied game instances to evaluate learning across repeated attempts and transfer to new situations. Findings show that retaining full action records aids learning, human players outperform agents, and the choice of harness significantly impacts performance and inference cost.

By Yibo Li, Jinhang Qiu, Zhi Zheng, Qianyun Guo, Jiaying Wu, Shuo Ji, Bryan Hooi
arXiv Machine Learning
Aug 27

Skill Issue: Are Skills Language-Invariant in LLMs?

The paper investigates whether large language models (LLMs) exhibit language‑specific skill differences by having two identical model instances play a text‑based game in different languages. Using a multilingual extension of TextArena, the authors evaluate three open‑weight models across eight languages and six games, finding that the same model can show markedly different performance—varying win–loss margins, invalid actions, and strategic choices—depending on the language interface. Analyses pinpoint language‑specific failures in spatial reasoning, card‑conditioned decisions, and optimal move selection, and demonstrate that adjusting the intermediate reasoning language can recover much of the lost performance.

By Bobby Cheng, Adam Gaber, Zhengyuan Liu, Catherine Arnett, Omer Goldman, Cheston Tan, Leshem Choshen