SocialArena is a benchmark designed to evaluate the social intelligence of large language models (LLMs) across three dimensions: role‑playing ability, mental state reasoning, and behavioral reasoning. It introduces a role‑playing dialogue paradigm with explicit character thinking to assess these capabilities in realistic human interactions, and a U‑TRIMF framework to synthesize diverse social contexts. The evaluation relies on a human‑annotated arena‑style protocol, supported by a $132.4K investment in recruiting and training annotators who compare model responses and provide preference labels and critiques.
By Zongqi Wang, Rui Wang, Yuchuan Wu, Hua Yang, Bingxue Qiu, Tianle Gu, Bing Zhao, Yujiu Yang, Jieping Ye, Yongbin Li
SciGame is a new benchmark that transforms scientific literature into structured cards, each providing both an Original and a Counterfactual environment for the same task. It evaluates language‑model agents in identification and discovery scenarios under Prior‑only, Fixed, and Adaptive protocols, using historical canonical laws and a live temporal collection. Experiments with five advanced models reveal that prior knowledge can hurt performance when relations change, adaptive experimentation benefits strong models but not weak ones, and accurate explanations may mask quantitative errors, highlighting the difficulty of recovering exact symbolic laws.
By Yusheng Zhao, Bowei He, Xue Liu, Mingxiao Li
The paper introduces prefix executability, a trajectory-level evaluation protocol for tool-using agents that checks whether every prefix of a predicted tool-call sequence remains executable under tool schemas, state preconditions, and data dependencies. It defines metrics such as Prefix Executability Curves, First-Error Depth, Trajectory Executability Rate, horizon-normalized AUC-PEC, and Recovery Rate to quantify reliability. Experiments on BFCL-v3, ToolSandbox, and $ au^3$-bench show that agents can achieve high final success while having low trajectory executability, revealing failure modes that final success metrics miss, and demonstrate that explicit pre-execution plan checking can improve prefix-level reliability without sacrificing terminal success.
By Amir H. Rezaeian, Weiyi Sun, Yassine Benajiba, Dan Roth
The paper reports on the Ramanujan Challenge, a benchmark of research‑level number theory problems, and evaluates submissions from AI models, CAS, and formal tools. It highlights that some correct solutions differ conceptually from official ones and even proves a published open conjecture. The authors introduce an AI‑assisted evaluation pipeline that significantly lowers false positives in natural language proofs and improves accuracy for supporting code, offering a scalable method for assessing high‑level mathematical work.
By Alan Sorani, Ashvni Narayanan, Michael Shalyt, Tali Monderer, Rotem Kalisch, Hila Barkan, Elyasheev Leibtag, Shachar Weinbaum, Carsten Schneider, John Campbell, Ido Kaminer
The paper introduces LLM-Integrated Bayesian State Space Models (LBS) for multimodal time-series forecasting, combining a state space model backbone with a pretrained large language model to handle both numerical and textual data. LBS allows flexible lookback and forecast windows, principled uncertainty quantification, and improved temporal generalization. Experiments on the TextTimeCorpus benchmark show a 13.20% performance gain over the previous state‑of‑the‑art while providing human‑readable forecast summaries.
By Sungjun Cho, Changho Shin, Suenggwan Jo, Xinya Yan, Shourjo Aditya Chaudhuri, Frederic Sala
Merge and Branch (M&B) is a tree‑structured supervised fine‑tuning method for large language models (LLMs) in recommendation systems. It compresses each historical item’s semantic identifier (SID) and text into a few learned tokens (Merge) and then uses the compressed history to predict multiple target SIDs (Branch), organizing each example as a shallow, wide tree. The approach includes a tree FlashAttention mechanism for efficient training and a tree‑structured key/value cache for constrained decoding, achieving better recommendation performance and improved training and decoding efficiency compared to strong baselines.
By Yunqin Zhu, Yao Xie
FilingLoop is an interactive benchmark and self‑evolving agent framework designed to evaluate and improve large language model agents in the context of corporate disclosure review and revision. It reconstructs 380 IPO review tasks from SEC correspondence, scoring agents against 6,099 requirements and auditing claims and revisions. The framework enables procedural learning across cases, converting feedback into role‑specific guidance without updating model weights, and demonstrates measurable gains in requirement fulfillment and comment resolution.
By Wentao Zhang, Yilei Zhao, Guangyan Gan, Shuo Sun, Lin William Cong
PairCLIP‑SWA is a single‑backbone deepfake detector that fine‑tunes an OpenCLIP ViT‑L/14 visual tower using a class‑balanced corpus created by CLIP‑similarity‑guided subsampling and then applies stochastic weight averaging over the fine‑tuning trajectory. Trained on the IEEE SP Cup 2025 (DF‑Wild‑Cup) benchmark, the weight‑averaged model achieves 97.04 % accuracy, 98.64 % AU‑ROC and a 4.07 % equal error rate—outperforming the best published three‑backbone ensemble. The study shows that averaging stabilizes validation performance, prevents misranking on disjoint test sets, and that CLIP pre‑training, rather than ensembling, is the key driver of accuracy.
By Md. Nakib, Sudipto Sarkar, A. J. A. Jubayer Talukder, Mohammad Ariful Haque
LLM-PowerTrace is an open dataset that provides time‑resolved, per‑GPU power traces for large language model fine‑tuning and inference. The traces are sampled every 100 ms across various NVIDIA platforms and include detailed workload and system metadata. The dataset covers a wide range of model families, sizes, and controlled sweeps over batch size, sequence length, precision, parallelism, checkpointing, concurrency, GPU frequency, KV‑cache and weight quantization, prompt and output lengths, and request‑arrival patterns, and it defines power‑grid‑facing metrics and an AI‑compute power & energy scaling law.
By Jiansheng Wang, Jaskeerat Singh, Haochen Xu, Xinyi Li, Mingyuan Yan, Shaohui Liu, Yuzhuo Li, Yize Chen, Wenqi Cui
WeVisCut is a long‑form script‑to‑video system that unifies narration, editing, and tool execution into a single agent. It uses a four‑stage data pipeline that reconstructs editing intent from finished videos, generating over 130K training examples to train a 9B policy capable of coordinating production and recovering from failures. The system introduces a 100‑task benchmark with five metrics, outperforming its base model, matching Claude Opus 5 in performance while using far fewer tokens, and surpassing task‑matched native editors on the proposed metrics.
By Chenyu Mu, Wenming Weng, Qinrui Wang, Xingsong Ye, Hao Yu, Jun He, Chong Sun, Chen Li, Jing Lyu, Erkun Yang, Cheng Deng
DeCoRL is a reinforcement‑learning framework designed to enhance multi‑image composition (MICo) by decomposing overall consistency into object‑level consistency. It first establishes instruction‑relevant object correspondences across reference images and localizes these regions in the generated image. A VLM‑based reward model, trained on single‑object consistency annotations, provides fine‑grained, object‑wise feedback that is aggregated into a multi‑object consistency reward to guide RL post‑training. The authors also introduce DeCo‑Bench, a benchmark covering compositions from 2 to 6 reference images, and show consistent improvements on both DeCo‑Bench and the public MICo‑Bench.
By Zhiqiang Wu, Shuang Sun, Jiale Zhang, Jing Li, Yibo Chen, Xu Tang, Yao Hu
TrussTutor-Bench is a benchmark of 200 verified two-dimensional truss problems with automatically generated student solutions, designed to evaluate large language models on solution accuracy, sycophancy, and error detection. Thirteen LLMs were tested, showing accuracy ranging from 64.5% to 97.0%, near-perfect error detection (90.4%–100%), and varying degrees of sycophancy (1.6%–50.0% of correct solutions rejected when a student doubted them). The benchmark highlights wide performance differences among models, especially in handling student doubt, and offers a verified, extensible standard for assessing technical reliability before deploying LLMs in educational settings.
By Evangelos Apostolakis, Erfan Hamdi, Sara Jahedazad, Emma Lejeune
MetaRefine is a lightweight, metadata‑aware score refinement architecture that uses original image geometry, multi‑level visual features, and a frozen model’s initial score to predict corrections. It improves correlation metrics (PLCC and SRCC) and reduces error metrics (MAE and RMSE) across 100 configurations on 21 datasets, with average gains of 7.55% and 7.10% and reductions of 23.27% and 21.93%, respectively. The method adds only a median 1.11% parameter overhead and can enhance state‑of‑the‑art assessment models on multiple benchmarks.
By Chuanzhi Xu, Chengkun Yue, Haolan Guo, Xuanhua Yin, Boyu Wei, Zihan Deng, Qiang Qu, Weidong Cai
The paper introduces neural triage decoding, a new framework for high‑rate quantum LDPC codes that adaptively uses neural message‑passing to predict error corrections, followed by iterative refinement and a rare diversification step. It demonstrates that the Triage decoder can handle ultra‑high‑rate codes encoding hundreds of logical qubits, achieving near‑optimal accuracy while reducing decoding latency by one to two orders of magnitude compared to existing decoders. The approach is grounded in a rigorous analysis of bitwise marginal decoding and shows robustness to mismatched error priors.
By Hong-Ye Hu, Zhenlai Wang, Binzhao Luo, Tao Xiang, Chen Zhao
Pharos-ESG is a unified framework that converts ESG reports into structured representations using multimodal parsing, contextual narration, and hierarchical labeling. It incorporates a reading‑order model based on layout flow, hierarchy‑aware segmentation guided by table‑of‑contents anchors, and a multimodal aggregation pipeline that turns visual elements into coherent natural language. The framework adds ESG, GRI, and sentiment labels and outperforms existing document parsing systems and general multimodal models, while also releasing Aurora‑ESG, a large‑scale public dataset of ESG reports with detailed layout and semantic annotations.
By Yan Chen, Yu Zou, Jialei Zeng, Haoran You, Xiaorui Zhou, Aixi Zhong
AEM: Adaptive Entropy Modulation for Multi‑Turn Agentic Reinforcement Learning proposes a supervision‑free credit assignment technique that adjusts entropy dynamics at the response level during RL training. By aligning uncertainty estimation with the granularity of LLM agent actions and rescaling advantages based on a response‑level uncertainty proxy, AEM improves the exploration‑exploitation balance without relying on dense intermediate supervision. Experiments on ALFWorld, WebShop, and SWE‑bench‑Verified show consistent performance gains across models from 1.5B to 32B, including a 1.4% improvement in a state‑of‑the‑art software‑engineering RL framework.
By Haotian Zhao, Songlin Zhou, Yuxin Zhang, Stephen S. -T. Yau, Wenyu Zhang, Lun Tian, Tianshu Zhu, Yifeng Huang, Yucheng Zeng, Jingnan Gu, Daxiang Dong, Jianmin Wu
ClueTracer is a training‑free, parameter‑free plugin that suppresses hallucinations in multimodal reasoning models by tracing how visual clues propagate from the question through the model’s reasoning pathway. It localizes task‑relevant image patches while ignoring irrelevant regions, thereby addressing the problem of reasoning drift where models over‑focus on question‑irrelevant entities. The method improves performance across various reasoning architectures by an average of 1.21× on reasoning benchmarks.
By Gongli Xi, Kun Wang, Zeming Gao, Huahui Yi, Haolang Lu, Qianshan Wei, Ye Tian, Wendong Wang
The System Hallucination Scale (SHS) is a lightweight, human-centered instrument designed to assess users’ perceived reliability of large language models across five dimensions: factual accuracy, source reliability, logical coherence, deceptiveness of presentation, and responsiveness to user guidance. It consists of ten balanced Likert-scale items modeled after the System Usability Scale and System Causability Scales, yielding a directional score and consistency diagnostic. In a supervised validation study with 193 participants, SHS showed strong internal consistency, high clarity, and took an average of 4.2 minutes to complete, revealing that users perceive source verifiability and presentation deceptiveness as more problematic than factual accuracy.
By Heimo M\"uller, Dominik Steiger, Markus Plass, Andreas Holzinger
GenoBERT is a transformer-based, reference‑free model that tokenizes phased genotypes and uses self‑attention to capture short‑ and long‑range linkage disequilibrium. Benchmarking on the Louisiana Osteoporosis Study and the 1000 Genomes Project shows GenoBERT outperforms four baseline methods, achieving an overall imputation accuracy of about 0.98 at up to 25% missing data and maintaining r² > 0.90 even at 50% missingness across multiple ancestries. The model’s 128‑SNP context window is sufficient to capture local correlation structures, and it remains robust to small sample sizes and weak LD, offering a scalable solution for genotype imputation without relying on reference panels.
By Lei Huang, Chuan Qiu, Kuan-Jui Su, Anqi Liu, Yun Gong, Weiqiang Lin, Lindong Jiang, Chen Zhao, Meng Song, Jeffrey Deng, Qing Tian, Zhe Luo, Ping Gong, Hui Shen, Chaoyang Zhang, Hong-Wen Deng
The paper argues that many newly proposed unsupervised feature selection methods are evaluated only against other methods without a proper baseline. Using random feature selection as a baseline, the authors demonstrate that several state‑of‑the‑art methods are actually outperformed in both performance and efficiency. They call for the inclusion of random selection as a baseline and for the development of better evaluation metrics to ensure realistic assessment of future methods.
By Muhammad Rajabinasab, Michael E. Houle, Oussama Chelly, Arthur Zimek