AutoTuneBench introduces a trustworthy measurement protocol for evaluating how large language model agents auto‑tune GPU kernels and serving engines. The benchmark addresses four failure modes—strawman baselines, machine‑dependent timing, saturated tasks, and infrastructure defects—by enforcing code‑frozen protocols, database validation, anti‑cheat checks, pre‑registered comparisons, and external result anchoring. Using this protocol, the authors demonstrate that previously reported speedups are inflated, revealing more modest improvements across different engines and machines.
By Li Chen
The paper introduces WFM, a Wiki Foundation Model designed to support complex agentic reasoning by combining dense document contexts with markdown-based topological linkages. It formalizes a Wiki Graph schema that preserves explicit topologies while embedding continuous semantics, and employs a query‑conditioned attentive aggregation for efficient message passing. The authors also propose an NCCL‑based protocol to reduce distributed system overhead, achieving a 10.5× training speedup and strong performance on long‑term memory and multi‑hop reasoning benchmarks.
By Junnan Dong, Linhao Luo, Senlei Zhang, Gong Chen, Taian Guo, Yifei Yu, Rong Tao, Tao Guo, Qian-Wen Zhang, Siyu An, Ruizhi Qiao, Xing Sun
The paper presents a systematic mapping of recent chess research involving humans, engines, neural and reinforcement‑learning systems, large language models (LLMs), and hybrid approaches. It identifies 84 core study families and classifies them by agent type, strategic‑reasoning stages, and evaluation dimensions, highlighting a strong focus on situation assessment, evaluation, and action selection while noting gaps in planning, explanation, metacognition, and human–AI collaboration. The study also distinguishes hybrid systems by integration timing and cautions that improved human performance in evaluations does not automatically imply human–AI synergy.
By Paolo Ciancarini, Remo Pareschi
The study investigates whether unusually high‑gain parameters in transformer models—specifically gated feed‑forward network (gated‑FFN) rows—play a critical functional role across both text and genomic foundation models. By computing exact bilinear weight operators and testing structural extremeness, the authors find that high‑gain rows are enriched for functional importance but do not reliably predict causal effect size or severity. The analysis reveals model‑specific causal organizations, including super‑additive interactions in DNABERT‑2 and position‑localized dependencies in GENERator, indicating that structural prominence signals enrichment rather than calibrated criticality.
By Alexandros Tzanakakis, Aris Karatzikos, Ilias Georgakopoulos-Soares
The paper investigates where the transition from memorization to generalization occurs within a Transformer. Using Transition Games—activation games with paired non-generalizing controls—it finds that distributed utility gain is driven by a block‑0 attention bias, with degree‑two modes explaining 67–92% of the contrast across replacement games. A detailed path study shows that block‑1 MLP mediates more of this effect than other downstream paths, contradicting the common "MLP memorizes, attention generalizes" hypothesis and suggesting that grokking is a spectral recoding of an existing distributed circuit rather than a module switch.
By Dekun Yang
The paper introduces MAGER, a multi-agent genetic evolution framework that automatically discovers meta-paths for large language models (LLMs) to reason about fake news propagation graphs. By compressing complex propagation structures into informative subgraphs, MAGER reduces modality mismatch and information overload, enabling frozen LLMs to perform structure-aware veracity reasoning. The authors also propose a graph in-context learning strategy that retrieves semantically and structurally similar demonstrations to enhance classification and reasoning, and report that MAGER significantly improves LLM performance in data‑efficient settings.
By Ziyi Zhou, Xiaoming Zhang, Hui Pang, Yuting Zhang, Tiesunlong Shen, Bingyu Yan, Erik Cambria, Litian Zhang
The paper introduces task‑weighted charts, a method that defines low‑dimensional coordinate systems based on a chosen functional of a language model’s representation. Using these charts, the authors show that next‑token prediction requires 70–90% of the residual stream’s width to maintain perplexity, a width largely unused by variance‑based analyses. The study demonstrates that charts trained under the functional’s metric preserve predictions better than variance‑based or optimal linear compression when only a few dimensions are retained.
By Alexandre Quemy
MUSE is a new benchmark designed to evaluate large vision‑language models on artistic image understanding within situated educational contexts. It separates image annotation from question generation, offering twelve tasks that cover visual perception, semantic and affective interpretation, cultural understanding, and compositional reasoning across diverse artistic images from Singaporean, Southeast Asian, and Western traditions. The benchmark reveals significant gaps in model performance, especially in affective interpretation and compositional reasoning, and highlights common failure modes for trustworthy educational multimodal systems.
By Luyao Zhu, Xun Wei Yee, Wei Li, Mun Thye Mak, Wee Siong Ng
The paper introduces DRAG, a query‑adaptive framework that jointly selects retriever and generator configurations for Retrieval‑Augmented Generation (RAG) systems. Two variants are presented: DRAG_QPP, a training‑free routing method using Query Performance Prediction and perplexity signals, and DRAG_SFT, a supervised approach that fine‑tunes an LLM to predict configurations. Experiments on three LLM families and four QA benchmarks show that DRAG_QPP matches strong static baselines while cutting inference latency, and DRAG_SFT consistently outperforms both static and training‑free adaptive baselines, demonstrating a better effectiveness‑efficiency trade‑off.
By Neeraj Anand, Payel Santra, Partha Basuchowdhuri, Debasis Ganguly, Sumit Bhatia
Lexara-RF introduces reference‑free metrics for evaluating conversational visual analytics agents that generate visualizations and natural‑language explanations. The framework uses only the prompt, data, and model response to score outputs, applying 13 metrics derived from visualization design theory and Gricean principles as consistency, intent‑alignment, and design validity checks. In tests against a human‑rated corpus, Lexara‑RF matches reference‑based methods, outperforms surface‑similarity NLG baselines, and accurately identifies structurally grounded failures.
By Srishti Palani, Vidya Setlur
The study evaluates gender representation in 8,000 images generated by four generations of the Stable Diffusion text‑to‑image model across 20 occupations and five prompt templates. It finds that 76.4% of the images depict male subjects, with 57.6% of historically female‑coded occupations also showing male subjects, and that newer model generations do not consistently reduce bias. Compared to U.S. Bureau of Labor Statistics data, the models underrepresent women by 20–46 percentage points, especially in near gender‑balanced fields such as scientists and cleaners.
By Shesh Narayan Gupta, Nik Bear Brown
VoiceTrace introduces a new benchmark, VoiceTrace-Bench, for hybrid speech retrieval that combines a textual query specifying "what" to retrieve with a reference speech specifying "who" to retrieve. The authors propose a two‑stage framework: VoiceTrace‑Emb, which learns unified audio‑text embeddings for efficient large‑scale retrieval, and VoiceTrace‑Reranker, which fine‑grains relevance by jointly examining query‑candidate pairs. Experiments show VoiceTrace outperforms existing methods on both traditional semantic speech retrieval benchmarks and the new hybrid setting.
By Aaron Yee, Fengjie Lu, Jiarui Hai, Chenang Jiang, Helin Wang, Siwei Tu, Weitao You, Lingyun Sun
ProgramDistill is a benchmark that evaluates coding agents by having them infer and implement features discovered through interaction with fully functional reference web applications. The benchmark is built by factorizing applications into features of varying granularity, each linked to replayable behaviors that can be executed via a gold patch. Using a mine‑craft‑patch pipeline, the authors discovered 1,975 replay‑verified behaviors across 26 applications and automatically constructed 4,063 tasks, enabling evaluation of nine frontier coding agents, including GPT‑6 Astra and Claude Opus 5, on both full‑application and partial‑application reconstruction tasks.
By Jeonghye Kim, Minseon Kim, Young Jin Kim, Matheus Pereira, Marc-Alexandre C\^ot\'e, Alessandro Sordoni, Xingdi Yuan, Zhengyan Shi
The paper introduces Label-Confidence-Aware Uncertainty Quantification (LCA-UQ), a method that uses Pointwise Kullback-Leibler divergence to align global entropy from multiple stochastic samples with the local confidence of a candidate answer. By bridging this gap, LCA-UQ improves the reliability and stability of uncertainty assessments in natural language generation. Experiments on popular LLMs and NLP datasets show that label sources significantly influence classification and that LCA-UQ outperforms existing uncertainty estimation approaches.
By Qinhong Lin, Yinglun Feng, Yuhao Zhang, Zhongliang Yang, Linna Zhou
The paper introduces ED$^2$, an end‑to‑end large language model–based sequential recommender that integrates index generation and recommendation through a dual dynamic index mechanism. It employs a multigrained token regulator for alignment supervision and custom instruction‑tuning tasks to capture high‑order user‑item interactions. Experiments on three public datasets show ED$^2$ outperforms baselines with an average 19.62% gain in Hit‑Rate and 21.11% in NDCG.
By Jun Yin, Zhengxin Zeng, Mingzheng Li, Hao Yan, Chaozhuo Li, Weihao Han, Jianjin Zhang, Ruochen Liu, Hao Sun, Weiwei Deng, Feng Sun, Qi Zhang, Shirui Pan, Senzhang Wang
CompArt introduces a new approach to aesthetic alignment in text-to-image generation by using the Principles of Art (PoA) such as Balance, Rhythm, and Emphasis to define explicit compositional constraints. The authors create a large dataset of 80,032 WikiArt images, each annotated with PoA analyses generated by a multimodal LLM, and present ArtDapter, a lightweight adapter that steers a pretrained diffusion model along ten PoA dimensions while preserving semantic fidelity. Experiments demonstrate that CompArt outperforms strong baselines in adhering to PoA controls under a dual evaluation protocol.
By Zhe Jin, Tat-Seng Chua
SurgRAW introduces a multi‑agent, chain‑of‑thought workflow for robotic surgical video analysis, leveraging a new SurgCoTBench benchmark with 14,256 QA pairs across five surgical tasks. The system uses an orchestrator to split scene understanding into two reasoning streams, panel‑discussion‑style collaboration among task‑specific agents, and retrieval‑augmented generation to incorporate surgical knowledge. Experiments show SurgRAW outperforms mainstream vision‑language models and a supervised baseline by 14.61% accuracy.
By Chang Han Low, Ziyue Wang, Tianyi Zhang, Zhu Zhuo, Zhitao Zeng, Evangelos B. Mazomenos, Yueming Jin
HyQuant introduces a hybrid-precision quantization framework for large language model (LLM) attention modules. It quantizes most attention states to low-bit formats while preserving a small set of vertical‑line tokens and local‑window states in full precision, guided by lightweight attention‑pattern signals. This design achieves near‑lossless accuracy across tasks while improving memory and hardware efficiency.
By Jiatong Ding, Bingxin Xing, Yu Zhang, Dian Ding, Xiaodong Yi, Xianbin Ouyang, Feihu Zhou, Kun Zhang, Zhenyu Guo, Hao Pan, Guangtao Xue, Yiming Zhang
The paper introduces ThoughtMed-1M, a large-scale medical visual question answering dataset built from de‑identified images and clinician‑generated commentaries, designed to capture structured clinical reasoning and image‑text alignment. Using this dataset, the authors train FOLTMed, a foundational large language model that achieves state‑of‑the‑art performance on 42 medical VQA benchmarks, with a macro accuracy of 85.4% and improved factuality and similarity metrics over existing models.
By Lingxuan Hou, Yuhua Xie, Yue Hu, Yan Zhuang, Junqi Li, Chengzhi Xia, Binh Phu Nguyen, Abubakar Siddique, Minh Nguyen, Yao Hou, Yanju Bao, Kexin Liu, Ke Chen, Jianjun Sun, Zeqi Li, Trung Nguyen, Jiangli Lin
The paper introduces ASTRIL-MPC, a language‑guided neural model predictive control framework that enables articulated tracked robots to navigate complex, contact‑rich urban environments such as stairwells and cluttered interiors. By combining a learned kinematics model that predicts short‑horizon state changes, an optimization‑based planner with multi‑objective costs, and a large language model that safely updates control weights, the system achieves up to 71% better traversal quality than non‑adaptive NMPC and 67% better than a PPO baseline, while eliminating collision impacts during descent. Real‑robot trials over four indoor obstacles confirm the method’s transferability to physical contact‑rich traversal.
By Zhenfeng Gan, Yanbo Chen, Lirong Che, Yongyi Ma, Rongkai Zhu, Xueqian Wang