SyntaxBench is a diagnostic benchmark and statistical evaluation framework for character‑level reasoning in large language models, comprising five core tasks—character counting, letter containment, palindrome detection, edit distance, and longest‑string selection—and a harder substring‑extraction stress test called index_to_span. The benchmark uses paired English and random‑string inputs, zero‑, one‑, and four‑shot prompts, and evaluates models from 2B to 32B parameters across multiple reasoning modes. It reports a wide range of metrics, including exact‑match and relaxed accuracy, Cohen’s kappa, McNemar tests, bootstrap confidence intervals, Kendall’s tau, class‑conditional metrics, tokenization analysis, and multiple‑comparison‑corrected tests.
By Mohsen Larni (Department of Computer Science, University of Nevada, Las Vegas), Sobhan Ebrahimi Azar (Department of Computer Science, University of Nevada, Las Vegas), Pouyan Nahed (Department of Computer Science, University of Nevada, Las Vegas), Kazem Taghva (Department of Computer Science, University of Nevada, Las Vegas)
AMBER is a new online, budgeted multi‑view reranking framework for vision‑language models that dynamically allocates computation to maximize information gain. It treats fragmented listwise VLM outputs as local tournaments and uses continuous Elo updates to maintain a lightweight global ranking state. Experiments on CIRR, CIRCO, and PhotoBench show that AMBER outperforms other multi‑call VLM reranking methods under comparable budgets, and remains effective even with lower budgets.
By Wenteng Chen, Jiachen Zhu, Rong Shan, Tianyi Xu, Yuxiang Chen, Congmin Zheng, Teng Wang, Junjie Wu, Weiwen Liu, Changwang Zhang, Weinan Zhang, Jun Wang, Jianghao Lin
The paper proposes a scalable traffic modeling approach that uses a single representative large language model (LLM) agent for each homogeneous traveler group, rather than one LLM per traveler. The representative agent maintains a mixed strategy over routes, updates it daily based on positive reinforcement signals, and uses a tunable step size to adjust its strategy. This design improves scalability, stabilizes learning, and produces interpretable dynamics that reproduce realistic behavioral patterns such as the decoy effect and income‑based willingness‑to‑pay differences.
By Hanlin Sun, Jiayang Li
Certified Mechanistic Edits: Behavioral Guarantees for Skill Removal and Preservation proposes a method to formally certify that disabling a specific circuit in a neural network removes a targeted skill while preserving another, for every input within a continuous region. The approach extends beyond testing by providing sound guarantees over continuous embedding-space regions, demonstrated on toy ReLU networks and a standard transformer. It also shows that no finite deterministic black-box test can guarantee removal, highlighting the necessity of formal verification.
By Md Sazid Uddin, Md. Khairul Alam Mazumder, M. F. Mridha
arXiv:2512. 19905v3 Announce Type: replace-cross Abstract: Recent developments in large language models have shown advantages in reallocating a notable share of computational resource from training time to inference time.
By Indranil Halder, Cengiz Pehlevan
The paper introduces COMETH, a framework that learns interpretable moral contexts from human judgments using probabilistic clustering and large language models. It builds a dataset of 300 scenarios involving six core actions and three moral rules, and employs an LLM-based preprocessing pipeline to cluster actions and scenarios. COMETH achieves roughly double the alignment with human judgments compared to end‑to‑end LLM prompting, while providing transparent explanations of the contextual features driving its predictions.
By Geoffroy Morlat, Marceau Nahon, Augustin Chartouny, Raja Chatila, Ismael T. Freire, Mehdi Khamassi
The paper introduces KD-Brain, a Prior‑Informed Graph Learning framework that incorporates semantic and clinical priors to model interactions among functional subnetworks in brain networks. It employs a Semantic‑Conditioned Interaction mechanism to guide attention queries by subnetwork identities and a Pathology‑Consistent Constraint to align learned interactions with clinical priors. KD‑Brain achieves state‑of‑the‑art performance on disorder diagnosis tasks and identifies biomarkers that align with psychiatric pathophysiology.
By Siyu Liu, Guangqi Wen, Peng Cao, Jinzhu Yang, Xiaoli Liu, Fei Wang, Osmar R. Zaiane
The paper introduces PALM, an algorithm that constructs a compact portfolio of aligned large language models (LLMs) capable of near‑optimal performance across a wide range of reward weightings for objectives like helpfulness, harmlessness, and conciseness. PALM uses a structured grid of weight vectors, lazy search, and pruning to efficiently identify a small set of policies that provably cover all target reward configurations within specified tolerances. Experiments demonstrate that PALM achieves smaller approximation gaps than portfolios built from uniformly spaced or randomly sampled weights and scales effectively to higher‑dimensional reward spaces.
By Cheol Woo Kim, Jai Moondra, Roozbeh Nahavandi, Andrew Perrault, Milind Tambe, Swati Gupta
DNAlign is a lightweight alignment framework that uses control‑theoretic optimization and null‑space projection to steer large language models toward safe behavior while preserving core knowledge and response quality. By treating the LLM as a dynamic system, it introduces controllable perturbations that are restricted to a harmful‑related subspace derived from neutral hidden states, and a value function trained on human preference data adaptively optimizes these control signals. Extensive evaluations across multiple LLM backbones show that DNAlign consistently reduces harmful outputs while maintaining fluency, coherence, and factual utility, outperforming prior alignment baselines without sacrificing generation diversity.
By Jisheng Dang, Yushuo Zhao, Dewei Liu, Junfeng Fang, Bimei Wang, Tiantian Rao, Hong Peng, Bin Hu, Tat-Seng Chua
The paper introduces Harness-Aware Distillation (HAD), a method for training smaller language model agents that preserves the surrounding harness—software managing context, tools, and feedback—while focusing distillation on the teacher’s contributions beyond the harness. HAD combines an action preference that contrasts teacher actions with and without harness information, and a validity check that filters out contradictory preference pairs. Experiments on long-horizon agent benchmarks show that HAD outperforms standard on‑policy distillation, reducing unproductive loops and improving error recovery without requiring task rewards or future information.
By Moonseok Choi, Taehong Moon, Giung Nam, Juho Lee
The paper introduces MITE, a method that transforms biomedical named entity recognition (BioNER) into a structure‑to‑structure generation task by encoding instructions and outputs in multiple programming languages (Python, C++, Java). This approach provides structurally diverse supervision without extra biomedical knowledge, and during inference it aggregates predictions via entity‑level voting to reduce language‑specific variance. Experiments on six BioNER datasets show that MITE outperforms BERT‑based and LLM‑based baselines and generalizes well across datasets.
By Songtao Li, Yijia Zhang, Jianyuan Yuan, Shidi Zhang, Fengyu Zhang, Hongfei Lin
The paper introduces CoC‑Seduce, a multi‑agent adversarial benchmark for evaluating how well large language models (LLMs) adhere to the rules of the tabletop role‑playing game Call of Cthulhu when acting as adjudicators. Using three LLMs to generate 5,376 samples across diverse settings and skill categories, the study benchmarks 22 target adjudicators and finds that newer releases and explicit reasoning do not guarantee robustness, with pseudo‑logic framing emerging as the most effective manipulation. The benchmark highlights the vulnerability of LLM adjudicators to rhetorical injection attacks that exploit narrative framing to bypass rule enforcement.
By Weiying Chen, Junlong Shen, Zhanyuan Guo, Xiaoou Zhou
The paper investigates whether large language model (LLM) agents that yield to a unanimous majority in multi‑agent debate truly change their underlying premise or merely adjust their statement. Using two‑hop factual questions with an unstated intermediate entity, the authors show that agents that concede still encode the original bridge in their internal representations, as revealed by a Jacobian‑lens analysis, even when the majority answer is wrong. Experiments across several open‑weight models demonstrate that hiding or removing the agent’s earlier answer increases conformity and that the original premise can be recovered from the question alone.
By Ziang Ni, Peng Zou
The paper presents a systematic study of how different compression techniques—pruning, quantization, and distillation—affect the capabilities of large language models (LLMs) in tasks such as mathematics, code generation, and question answering. It introduces a framework that measures capability loss and relates it to factors like model size, training stage, and compression settings, yielding simple predictive relations that generalize across unseen configurations. The authors demonstrate that sharing density responses across pruning levels can dramatically reduce the number of measurements needed, and that their predictive models closely match regression results while offering efficient decision guidance for compression method selection.
By Xueqi Cheng, Liang Wu, Kelly Wan, Liangjie Hong, Yushun Dong
The paper investigates why increasing batch size during large language model pretraining helps, challenging the common assumption of bounded stochastic gradient variance. By introducing a generalized Blum–Gladyshev noise model that allows variance to grow with a tunable exponent, the authors derive theoretical bounds on oracle complexity and design an adaptive batch scheduler that adjusts batch size dynamically. Experiments on OLMo2 models up to 1B parameters show that this scheduler achieves lower validation loss than fixed small or large batch training while using fewer iterations.
By Arda Fazla, Antesh Upadhyay, Ege C. Kaya, M. Berk Sahin, Abolfazl Hashemi
PowerBench is a new evaluation framework that measures how language models respond to power‑shifting requests, distinguishing self‑empowerment, disempowerment, and power grabbing while controlling for neutral requests. The authors built and released a dataset covering different power domains, contexts, scales, and prior power standings, and tested 24 models from US and Chinese developers across three experimental conditions. Results show models are more likely to refuse power‑grabbing than disempowerment, and that biases exist toward helping US users gain power while resisting US users taking power from others, with additional effects when the user is an AI agent or when requests are in different languages.
By Nicolas Martorell, Wendy Brau, Gonzalo A. Heredia, Tom\'as Pablo Korenblit, Gaspar Labasti\'e, Tom\'as Gimenez Molina
Tropical Reinforcement Learning replaces the traditional sum of probabilities with a maximum operation, forming a tropical semiring. This change allows the value of a state to reflect the log-probability of its most likely verified solution and provides an explicit path that can be replayed. The proposed TROPIC algorithm, applied to deterministic, resettable environments, outperforms strong on‑policy baselines on tasks such as Sokoban, Countdown, FrozenLake, and WebShop.
By Arip Asadulaev, Aladin Djuhera, Karim Salta, Holger Boche, Fakhri Karray, Martin Takac
The paper introduces DiffGCMS, a discrete graph diffusion model that generates molecular structures directly from GC–EI–MS spectra, and couples it with a large language model for post‑processing. In the first stage, DiffGCMS produces candidate structures; the LLM then validates, repairs, and reranks these candidates while offering interpretable explanations of fragment‑ion peaks. On a large NIST 20 test set, the combined approach improves accuracy and ensures 100% candidate validity for small molecules, demonstrating the value of spectrum‑aware post‑processing.
By Changlin Liu, Tianyu Yi, Chengchun Liu, Boxuan Zhao, Fanyang Mo
EviDent-CBCT is a framework that generates dental cone‑beam CT reports from limited clinical data by first extracting a discrete record of tooth‑level, global, and tooth‑IAC evidence using an anatomy‑aware network. The evidence is reconciled through a dental‑logic consistency projection before being rendered into a report by a deterministic renderer and an image‑blind language model. The system achieves higher evidence set‑F1 and logical‑F1 scores than baselines and ranked highly in the ODIN 2026 challenge, demonstrating the effectiveness of a discrete evidence record for auditable CBCT report generation.
By Ruiyang Hao, Zhi Qin Tan, Yulan He, Owen Addison, Yunpeng Li
RxnOptBench is a new benchmark that tests large language models on the task of selecting optimal reaction conditions—such as catalyst, ligand, solvent, temperature, and atmosphere—by reading real condition‑screening tables from 2025 organic‑methodology papers. The benchmark uses a continuous relative score that combines yield with stereochemical metrics (ee, dr, rr) and includes a design that separates in‑context literature use from mere memorization. Evaluation across nine frontier LLMs and three chemistry‑specialized LLMs shows that even the best models still have significant room for improvement, with chemistry‑specialized models performing at random levels on multi‑axis selection while open‑weight models approach proprietary models.
By Lingli Ge, Yubin Wang, Junyuan Gao, Jiahe Song, Jiaxing Sun, Boyu Zhu, Haote Yang, Jingchao Wang, Lixin Ma, Jiang Wu, Yuqiang Li, Conghui He