The paper introduces DIAL, a framework that uses large language models (LLMs) as judges while mitigating position bias and aligning their judgments with human preferences. DIAL separates judge‑specific position effects, learns shared structure in debiased LLM preferences, and adaptively calibrates this structure toward human targets using limited human comparisons. Experiments on simulations and three human‑preference benchmarks show that DIAL remains robust to unbalanced response order, achieves strong human‑aligned rankings with few labels, and adapts when LLM information is imperfect, supported by a real‑data study of over 410K judgments from 21 LLM judges.
By Zesheng Cai, Yingqi Fan, Sichang Chen, Jin-Hong Du
The study investigates how the choice between processing page images or parsed text in an information extraction pipeline depends on the document’s layout, focusing on privacy‑sensitive, on‑premise scenarios with small models (≤8 B parameters). It evaluates accuracy and energy consumption across input representations, model families, and inference settings, finding that batching dramatically reduces energy use, FP8 quantization offers modest savings, and neural OCR is far more energy‑intensive than classical OCR. The optimal representation varies: vision‑language models excel on layout‑rich documents, while small text‑only models with a cheap parser perform best on near‑plain‑text contracts, achieving higher accuracy and lower energy than any vision‑language setup.
By Christoph Walser, Mauricio Fadel Argerich, Jonathan F\"urst
Game Arena is an open, continuously expanding platform that evaluates large language models through competitive games, allowing head‑to‑head matchups in structured environments. Unlike static benchmarks, it prevents performance saturation by increasing gameplay difficulty as models improve. The report outlines the infrastructure and presents three pilot games—Chess, Poker, and Werewolf—covering perfect information, imperfect information, and multiplayer settings, and details evaluation metrics and competition results.
By Bovard Doerschuk-Tiberi, Yao Yan, Justin Chiu, Hann Wang, Timothy Chung, Martyna Plomecka, John Schultz, Jon Lipovetz, Clayton Drazner, Yuchen Zhuang, Jaimie Hwang, Nate Keating, Riley Jones, Andrew Lee, Oran Kelly, Ian Gemp, Michael Aaron, Laurel Prince, Kate Larson, Jeff Moser, Harrison Jobe, Chad Woodford, Siqi Liu, Andrew Wang, Bo Chang, Christopher D'Mello, Diane Chaleff, Addison Howard, Johnny Yip, Chuck Sugnet, Antonio Gulli, Meghan O'Connell, Will Cukierski, Nenad Tomasev, Dima Yeroshenko, Kinjal Parekh, Roxanne Daniel, Marc Lanctot, Domino Weir, Elsa Dong, Daniel Hennes, Melissa Nalubwama, Robert Fraser, Ryan Trostle, Jun Peng, Tom Mason, Lloyd Hightower, Chiamaka Chukwuka, Yuexiang Zhai, Phoebe Kirk, Yi Su, Yuting Han, Jie Ren, Chris Prichard, Sahand Sharifzadeh, Karim Hakimzadeh, DJ Sterling, Meg Risdal, Kate Olszewska, Ya Xu, Orhan Firat, Minmin Chen
The paper introduces Steiner’s taxonomy of group tasks to study how multi‑agent large language model (LLM) teams scale on disjunctive versus compensatory tasks. By modeling agents as conditionally independent given the item, it shows that plurality voting converges to the modal answer while averaging converges to the item‑level bias. Experiments with 13 open‑weight models and up to 30 agents reveal that disjunctive tasks benefit from larger teams, whereas compensatory tasks like Fermi estimation see little improvement, highlighting that task structure and aggregation method fundamentally determine team scaling.
By Carolina Fortuna, Blaz Bertalanic
DeepEdu‑v1 is an AI‑tutoring system tailored for Vietnamese education that addresses data‑sovereignty and local curriculum alignment issues. It uses a long‑context inference engine to reduce retrieval calls and prefill latency by about 35%, and a self‑improving agentic layer that curates verified local knowledge without fine‑tuning. In deployment, DeepEdu achieves nearly twice the speed of standard vLLM serving and raises agentic accuracy from 70.0% to 79.5% on complex tasks, especially in financial reasoning and interactive‑agent benchmarks.
By Quang Nguyen, Hieu Nguyen, Hien Hoang, Toan Pham, Cong Tran, Nam Vu
The paper demonstrates that fine‑tuning reasoning models to predict their own confidence at intermediate steps—using only 600 self‑supervised examples—substantially improves inference efficiency. Without adding any explicit stopping or length penalties, the models generate up to 25 % fewer tokens while maintaining accuracy on mathematical, scientific, and coding benchmarks across several architectures. The study finds that confidence supervision preserves the models’ high‑level reasoning structure rather than merely suppressing specific behaviors.
By Parsa Hosseini, Akasha Tigalappanavara, Sumit Nawathe, Chenrui Fan, Sourya Basu, Genta Indra Winata, Anirban Das, Soheil Feizi, Nima Chitsazan
ENAS is a hardware‑aware neural architecture search framework tailored for TinyML on microcontrollers. It uses a static feasibility check, a cell‑based search space with various block types and skip connections, and a three‑stage hybrid search strategy (random → top‑K → mutation) with cross‑run caching. The framework runs efficiently without GPUs, achieving significant search‑time speedups and competitive accuracy on Visual Wake Words and Melanoma Cancer benchmarks across a range of microcontrollers.
By Mohd Moin Khan, Naman Srivastava, Pandarasamy Arjunan
The study investigates which neurons in a frozen BERT-base-uncased encoder support AI‑text detection using the RAID benchmark across six generators. By applying an L1‑to‑L2 sparse‑probing protocol to all 9,216 CLS hidden‑state dimensions, the authors identify a stable set of fewer than 1% of neurons per generator that largely preserves detection accuracy. Bidirectional activation patching confirms the causal relevance of this set, while mean‑ablating the neurons shows the signal is redundantly distributed, and cross‑generator analysis reveals a bipartite structure with instruction‑tuned generators concentrating more stable neurons in the final layer.
"whyItMatters":"The findings demonstrate that a small, stable subset of BERT neurons can reliably support AI‑text detection across diverse generators, enabling efficient detector design without re‑identifying neurons for each new generator."
By Pawe{\l} Blicharz, Mi{\l}osz Grunwald
The article surveys fake review detection research, focusing on how pre‑trained language models (PLMs) and large language models (LLMs) influence both the generation of deceptive reviews and their detection. It reviews 211 studies from 2018 to early 2026, categorizing methods by evidence source—such as review text, sentiment, rating behavior, temporal metadata, user‑product graphs, multimodal content, external knowledge, and LLM‑generated signals—and by fusion level. The survey traces the evolution from traditional machine learning to PLM‑based and LLM‑based approaches, evaluates performance on Amazon, Yelp, and OpSpam benchmarks, and highlights open challenges including adversarial generation, cross‑domain transfer, uncertainty‑aware fusion, robustness to missing sources, interpretability, and trustworthy evaluation of AI‑generated deceptive content.
By Fanji Yang (Guizhou University of Finance and Economics), Huiyao Chen (Harbin Institute of Technology), Xi Yu (Guizhou University of Finance and Economics), Meishan Zhang (Harbin Institute of Technology), Xiaohong Xiao (Guizhou University of Commerce), Mingsen Deng (Guizhou University of Finance and Economics)
SignTrace is a system that tackles the reverse‑lookup problem in Chinese sign language by allowing users to describe a sign in natural language and retrieve the corresponding entry from a dictionary of 6,699 signs. It combines large‑language‑model–based dictionary enrichment, action extraction, seven‑channel retrieval, and candidate reranking, achieving a 94.0% Hit@1 and a mean reciprocal rank of 0.9540 on a benchmark of 500 queries. The system has been deployed for user trials, receives positive informal feedback, and processes median queries in 13.37 seconds.
By Zengji Tu, Xingye Zhu, Ningjing Wang, Tingyi Huang, Yangjunfeng Zhu, Dai Wan
The paper "Strategic Self-Consistency" investigates how large language model providers might exploit the self‑consistency technique—generating multiple reasoning paths and selecting the majority answer—to overcharge users. The authors present a simple, efficient algorithm that strategically generates and reorders extra reasoning paths so that each appears necessary for the majority vote, thereby evading detection by auditors. Experiments on Llama, Qwen, and DeepSeek-R1 models across math, science, and QA benchmarks show that the added paths follow a heavy‑tailed distribution and that significant overcharging can persist even under stringent audits with a false‑positive rate below 0.1.
By Tori Qiu, Ander Artola Velasco, Manuel Gomez-Rodriguez
The paper introduces ContextXR, a benchmarking framework designed to evaluate context‑aware interfaces in Extended Reality (XR) systems. It models an XR application as a graph of functional facets—semantically coherent groups of capabilities that support a shared user intent—and uses this representation to build MineXR++, a dataset enriched with facet‑level annotations. The framework defines three core tasks—context factor analysis, initial facet suggestion, and next facet suggestion—and evaluates suggestion methods via a simulated interaction metric that measures navigation and search cost to reach desired functionality.
By Hyunsung Cho, Sarah Yewon Yun, Nancy Ruonan Sun, Ben Lafreniere, Mark Parent, Kashyap Todi, Tanya R. Jonker, Hrvoje Benko, Sherry Tongshuang Wu, David Lindlbauer
The paper introduces KNOWS, a benchmark for evaluating web agents that act as assistants by retrieving, synthesizing, and presenting information across complex, multi-step browser tasks. It outlines a task design rubric, evaluation protocol combining deterministic checks with LLM judgments, and reports that current agents achieve only modest success, with the best performing agent succeeding on less than 3% of tasks. The study highlights significant gaps in agents’ tool use, visual understanding, and long‑horizon reasoning.
By Alexander Gill, Md Farhan Ishmam, Xuyen Nguyen, Neha Bhat, Parker Henry DeYoung, Fateme Hashemi Chaleshtori, Nathan Stringham, Kenneth Marino, Ana Marasovi\'c
TrafficImag is the first benchmark designed to evaluate counterfactual roadside traffic video generation, combining a large roadside dataset with an executable protocol that supports behavior reasoning, intervention-aware image editing, and conditional video generation. Each intervention is encoded as an actor-level program specifying target actor, intended behavior, legal route, interaction order, and temporal constraints, allowing a unified evaluation across diverse foundation models. The benchmark assesses four validity dimensions—initial-state correctness, route and behavior validity, interaction consistency, and non-target preservation—and reports that the best models achieve 80.4% macro F1 for reasoning and 55.0% end-to-end success when using a complete condition interface.
By Xiangyu Li, Tianyi Wang, Zhihao Dou, Christian Claudel, Zhaomiao Guo
The paper introduces LIRSeg, a method that replaces explicit Chain-of-Thought reasoning in multimodal large language models with a compact set of learnable latent tokens for reasoning segmentation. LIRSeg is trained in two stages—spatial alignment and GRPO—while employing extreme-advantage sampling, decoupled exploration-stability updates, and latent diversity amplification to enhance token informativeness. Experiments show that LIRSeg improves segmentation accuracy and reasoning efficiency, achieving significant gIoU gains over the VisionReasoner baseline and reducing reasoning tokens by about 16×.
By Tianhang Guo, Yulin He, Wei Chen, Wenjuan Zhou, Yuhang Li, Xinbiao Gan
The paper introduces iDriveVLA, a multi‑modal planning framework for autonomous driving that addresses a generation‑evaluation asymmetry by improving candidate trajectory spaces and providing a unified, safety‑aware evaluator. It combines a Safety‑aware Scorer for risk estimation with a VLM‑guided Modulator that adapts weighting to the scene, and employs an oracle‑aligned progressive training strategy. On the NAVSIM v1 leaderboard, iDriveVLA achieves a new state‑of‑the‑art PDMS score of 94.95, surpassing human‑expert performance.
By Zeyu He, Shiqi Liu, Ke Chen, Yun Yan, Jinzi Wu, Dianqiao Lei, Sirui Wang, ShuRui Peng, Tao Chen, Zhuo Huang, Yu Wu, Yadong Shao, Zhichao Li, Ke Sun, Yang Guan, Keqiang Li, Shengbo Eben Li
The paper introduces Persistent Negative Adversarial Distillation, a method that improves black-box on-policy distillation by maintaining a live pool of historical teacher–student comparisons to stabilize the discriminator’s negative distribution. By anchoring the discriminator with these persistent negatives, the approach reduces reward-estimation error and yields smoother, higher-performing student policies across multiple benchmarks. The study demonstrates that the choice of negative samples is a critical design factor in effective black-box distillation.
By Haixu Ma, Saad Lahrichi, Weiwei Li, Kevin Han, Weiqiang Wu, Peggy Yang, Dongzhuo Li, Ruiyi Li, Serena Li, Gedi Zhou, Mingze Gao, Abhishek Kumar, Xiangjun Fan, Lizhu Zhang
The paper introduces a unified evaluation protocol for robust counterfactual explanations (CFE), testing six robust methods and two baselines across four tabular datasets under eight types of model change. It shows that robustness scores vary by change type and that methods designed for one change family may not transfer to others, with RobX performing most consistently. The study emphasizes the need for a common protocol that defines model changes, measures their impact, and separates generation performance from robustness.
By Marcin Kostrzewa, Maciej Zi\k{e}ba
UltraG-Bench is a large‑scale, multi‑task benchmark designed to evaluate pixel‑level evidence grounding in ultrasound images. It comprises 40 public segmentation datasets covering 13 anatomical categories and includes three progressive tasks—instruction‑guided segmentation, evidence‑grounded VQA, and evidence‑grounded report generation—with a total of 736,726 annotations. Evaluation of 14 state‑of‑the‑art models shows a significant gap between semantic understanding and fine‑grained pixel‑level localization, and the authors propose UltraG‑Agent, which combines a vision‑language model with the ultrasound‑specific segmentation model UltraSAM to improve both semantic prediction and visual grounding.
By Quanhao Zhu, Bo Xu, Rui Lin, Chenyuan Wang, Yu Shao, Boling Zhu, Jiuyan Sun, Liang Zhao, Hongfei Lin, Feng Xia
MVVBench is a new benchmark for multi‑view video reasoning that tests vision‑language models on tasks requiring integration of spatial and temporal evidence across multiple, often non‑overlapping camera streams. The benchmark contains questions that cannot be answered from any single view or single moment, forcing models to jointly reason across views and time. It evaluates six capabilities—including attribute identification, relative distance, camera pose, and compositional counting—and provides human‑authored QA, rigorous verification, and detailed error analysis.
"whyItMatters":"The benchmark offers a rigorous evaluation of 4D multi‑view reasoning and a foundation for future progress toward reliable embodied perception."
By Hyungjin Chung, Byeongjun Park, Joonseok Lee, Hojun Kim, Jaeho Choi, Byung-Hoon Kim