UXBench is the first user‑centric benchmark that uses real user feedback to evaluate AI assistants. It comprises three tasks—UX Judge, UX Eval, and UX Recovery—built from 7,400 test instances drawn from over 70,000 interaction logs of a mainstream Chinese AI assistant, covering 8 scenarios and 83 domains. Experiments on 26 state‑of‑the‑art language models reveal how model capability affects user experience and highlight six key findings about user feedback prediction and engagement.
By Mengze Hong, Xia Zeng, Zeyang Lei, Sheng Wang, Chen Jason Zhang, Di Jiang, Taiming Fu, Jinfeng Huang, Mengqiao Liu, Qinghe Chang, Haosheng Zou, Qiongyi Zhou, Sijun He, Simonjmdeng, Minlong Peng, Di Liang
Switch‑SWE is a new benchmark that tests whether coding agents can resume an unfinished task after working on other tasks from the same repository. It provides 200 focus tasks from 19 Python repositories, each paired with matched uninterrupted and switched sessions, and evaluates agents on the final patch only. The benchmark reveals that interleaved work creates an average 11.5‑percentage‑point gap in task resolution, and a context‑management plug‑in can improve performance by 5.5 to 15.5 percentage points.
By Shuochen Chang, Qianli Ma, Yibo Miao, Qingyang Liu, Bingjie Gao, Haonan Zhao, Siyu Sun, Yulin Sun, Han Zhao, Jiangtong Li
MedPsy introduces a family of 1.7B and 4B text‑only medical language models optimized for edge deployment, combining synthetic medical data, chain‑of‑thought targets from a 235B teacher, and a four‑stage post‑training curriculum. On seven closed‑ended medical benchmarks, MedPsy‑4B outperforms MedGemma‑27B while being 6.75× smaller, and MedPsy‑1.7B surpasses MedGemma‑1.5‑4B by 11.42 points despite being less than half its size. The models also reduce response length, disk footprint, and end‑to‑end latency on real‑device inference, keeping peak RAM below 5 GB on laptops and smartphones.
By Davide Vitabile, Alexandro Buffa, Akshay Nambiar, Amril Nazir
TAC-CoT is a framework that replaces explicit chain-of-thought reasoning with compact latent reasoning segments, keeping only textual tokens for tool calls, observations, and final answers. This interleaved design allows large language model agents to use tools without generating full verbal reasoning steps. Experiments on math and QA benchmarks show up to 8.8‑point accuracy gains and a 6.8× reduction in model forward passes, while also improving tool‑use quality in single- and multi‑tool settings.
By Juseon Do, Sangmin Bae, Jeesu Jung, Hwanjun Song
HaloGuard 2.0 is an open‑weight, streaming output guard that assigns a safe/unsafe probability to each response token in a single forward pass, allowing deployments to halt a stream once harm is detected. It builds on HaloGuard 1.0’s input‑guard constitution, training on a 478,532‑response corpus and a second corrective phase with 106,043 counterfactual responses to address a shortcut that previously caused the model to judge prompts instead of responses. The resulting 0.8 B‑parameter model achieves an 83.6 macro F1 on six English safety benchmarks—up 10.8 points over the prior version—while using 5–10× fewer parameters than the strongest full‑response guards, and its multilingual variant reaches 83.1 F1 across 17 languages.
By Ashmiya Lenin, Navaneeth Sangameswaran, Preetham S
The paper presents a multilingual audit of stereotype bias in large language models (LLMs) using 12,000 locally written Reddit posts in English, Spanish, French, and German. It evaluates gender, racial, and religious bias across six models (three open-weight and three proprietary APIs) with a two-blank design to reduce grammatical cues, finding that religion shows the highest imbalance, Spanish leads in gender bias, and German in race and religion. The study highlights that aggregate scores can mask category- and context-specific biases, underscoring the need for detailed inspection before deploying LLMs in consequential settings.
By Zihao Sun
The paper introduces an expectation‑observation consistency verification (EOCV) mechanism for large language model (LLM) planning. EOCV allows batched executors to predict multiple actions in one model call while still grounding each step by comparing expected outcomes with actual observations. When consistency holds, execution continues; mismatches trigger early termination and replanning. Experiments on text‑based and multimodal benchmarks show that EOCV matches reactive executors in task performance while cutting weighted token usage by 29‑35% and works across different model backbones.
By Qingfu Zhu, Huanyang Zheng, Ting Liu, Wanxiang Che
The paper introduces Adaptive Index Reuse (AIR), a method that dynamically determines how many layers of a sparse attention indexer should refresh during a language agent’s reasoning process. AIR replaces the static layer‑refresh schedule of IndexCache with a small controller that selects from a nested library of patterns based on recent computation signals and tool events, optimizing for task reward under a compute budget. Experiments on Qwen3‑4B‑Instruct‑2507 with a 128K context show AIR improving success rates on SWE‑bench Verified, Terminal‑Bench 2.0, and three τ²‑bench domains while reducing time per successful task, and it matches or surpasses the static schedule across 50 varied settings.
By Guangyuan Wang, Yifan Zhang, Shilong Liu
The paper investigates whether large language model (LLM) agents can acquire new knowledge from tutorial videos without updating their parameters. It introduces Tutorial Agentic Learning (TAL), a non‑parametric continual‑learning framework that builds a structured tutorial‑book memory via a two‑stage adversarial process, and retrieves information through a Table‑of‑Contents tree during inference. On the VidLearn multimodal QA benchmark—comprising 72 chess tutorial videos—TAL outperforms baseline models, achieving up to 113% relative gains on training data and 91% on test data, and improves in real‑world chess openings in 69% of head‑to‑head games, with additional evidence of generality in a second domain.
By Haoyu Zhao, Zafeirios Fountas, Jun Wang
Poolwise is a protocol that evaluates reward models for selecting training data across different prompts by choosing a fixed fraction of responses based on score and measuring the quality of the selected set with external labels. It reports within‑prompt ranking separately and shows that training each prompt’s mean score to match its average response rating improves selection over permuted or omitted mean targets in all tested seeds at 30% retention. In grouped HelpSteer experiments, the model with the highest within‑prompt accuracy selects a lower‑rated set than the correct‑mean model, and subtracting prompt means from reference responses preserves local rankings but reduces selected helpfulness for several reward models and judges.
whyItMatters":"Poolwise offers a direct method to compare reward models and score transformations for data selection, addressing the limitation of existing benchmarks that only compare responses within the same prompt."
By Mateusz Modrzejewski, Tomasz Radzikowski
Decentralized Multi-Agent Coordination and Communication in Shared Codebases introduces ParallACT, a benchmark for studying how coding agents with independent, human‑driven requests interact in shared repositories. The benchmark includes complementary tasks that require agents to cooperate via a shared interface, and contradictory tasks where agents’ edits conflict, leading to poor merged outcomes. Experiments show that a single‑agent oracle outperforms parallel agents, especially on contradictory tasks, and that merely notifying agents of each other or providing a communication channel only marginally improves performance at a token cost.
By Rishabh Mediratta, Lingyun Xiao, Peter Stone, Joydeep Biswas, Elias Stengel-Eskin
NitiBench+ is a Thai legal question‑answering benchmark that spans 17 domains and contains 1,670 items derived from real court decisions. Each item includes both the law text that was in force at the decision date and the current text, along with a retrieval corpus of every dated version of every Thai statute section. The benchmark evaluates models using the Thai Bar Council rubric, Bar‑standard score, and citation overlap, and it reveals that reading outdated or missing law significantly degrades performance.
By Pawitsapak Akarajaradwong, Sarana Nutanong, Chompakorn Chaksangchaichot
arXiv:2611.03179v1 Announce Type: cross
Abstract: People often attach instructions to their search queries, asking for pages written for beginners, documents in a certain language or official sources...
By S. M. Mahbub Uddin Ahmed, Abdullah Al Musaddiq
arXiv:2611.03308v1 Announce Type: cross
Abstract: A frontier language model was recently reported to solve 1D NMR structural elucidation without tools. We ask whether this capability transfers to tan...
By Gabriel Asher, Cailum M. K. Stienstra, Tomo Oga, Devesh Shah Antonio H. de O. Fonseca, Niall O'Connor, Michael Widrich
arXiv:2611.01728v1 Announce Type: cross
Abstract: Large language model (LLM)-based multi-agent systems (MAS) tackle challenging tasks by dividing work among agents and coordinating their contribution...
By Zerui Chen, Qinggang Zhang, Qichuan Liu, Zhishang Xiang, Zhimin Wei, Yunbo Tang, Zhuo Li, Jinsong Su, Zhihong Zhang
arXiv:2604.24693v2 Announce Type: replace
Abstract: Linear activation steering is a powerful approach for controlling and specializing large language models by adding linear representations of concep...
By Brandon Hsu, Daniel Beaglehole, Adityanarayanan Radhakrishnan, Mikhail Belkin
arXiv:2611.03171v1 Announce Type: cross
Abstract: On July 1, 2025, Vietnam reduced its 63 provinces to 34, abolished its 696 districts, and replaced its 10,035 commune-level units (wards) with 3,321....
By Phan Binh Nguyen Lam, Nguyen Thai Anh
arXiv:2503.01036v3 Announce Type: replace
Abstract: We introduce a novel kernel-based framework for learning differential equations and their solution maps. The framework is efficient in its data req...
By Bamdad Hosseini, Alexander Hsu, Yasamin Jalalian, Juan Felipe Osorio Ramirez, Houman Owhadi
arXiv:2609.38850v2 Announce Type: replace-cross
Abstract: Reasoning models are trained to produce correct answers, yet selective deployment requires ranking which answers to accept or defer, and outc...
By Zhimin Gao, Meibo Hu, Jiamian Wang, Zhiqiang Tao, Pichao Wang
arXiv:2610.18541v2 Announce Type: replace
Abstract: Language models are increasingly expected to support users in stressful or uncertain situations, where warmth and empathy are desirable. While rece...
By Letian Shi, Kanwal Mehreen, Deheng Zhang, Yuxia Wang