The paper introduces point‑in‑time language models that are trained only on text available up to each calendar date, thereby eliminating lookahead bias that plagues standard large language models. By scaling decoder‑only transformers to 4 billion parameters and training on 1 trillion chronologically filtered tokens from FineWeb, the authors produce monthly checkpoints from 2013 to 2024 that achieve performance close to unconstrained models of similar size on common‑sense reasoning and language understanding tasks. Instruction fine‑tuning with LoRA further enhances downstream usability, and the authors release the full pipeline—including dataset construction, training infrastructure, and evaluation code—for reproducible research.
By Bryan Kelly, Semyon Malamud, Johannes Schwab, Teng Andrea Xu
ChannelGuard introduces an application‑controlled defense that inspects information at every handoff between models in multi‑agent LLM systems, eliminating the need for extra training or model calls. In over 5,700 test runs across five agent models, it stopped all attacks hidden in tool outputs, and on two open models it reduced successful prompt attacks from 80% to 3% and from 77% to 10%, without affecting accuracy on 30 math problems. The study shows that safety scores can mask very different underlying protections, and that relying solely on model or provider filters may be risky if either changes.
By Elias Hossain, Md Mehedi Hasan Nipu, Fatema Tuj Johora Faria, Shubhashis Roy Dipta, Maleeha Sheikh, Tasfia Nuzhat Ornee, Niloofar Yousefi
The paper introduces the Directional Influence Function (DIF), a new estimator for measuring how training samples affect the solution of constrained learning problems. DIF incorporates constraints directly by formulating optimality conditions as a variational inequality, thereby avoiding the bias seen in classical influence functions. Experiments on constrained linear regression and fairness‑constrained CNNs show that DIF accurately predicts changes in test loss and aligns closely with actual retraining results.
By Xin Wang (Jeff), R. Tyrrell Rockafellar (Jeff), Xuegang (Jeff), Ban
The study examines how large language models (LLMs) decide who gets access to restricted academic resources in a controlled simulation. Models consistently favor PhD candidates, but their preferences shift with the requester's region and academic status. Large proprietary LLMs tend to favor low‑income or Global South applicants, while smaller open‑weight models favor the Global North, suggesting that model scale, alignment, and training data influence gatekeeping biases.
By Nouar AlDahoul, Hezerul Abdul Karim, Myles Joshua Toledo Tan
The paper introduces AVCE, a method for verifiable concept erasure in diffusion models that audits latent representations to identify vulnerable directions and then edits cross‑attention and self‑attention projections in closed form. By fine‑tuning with pathway‑level auditing losses and orthogonal gradient projection, AVCE reduces attack success rates by 5.07× and improves auditing scores by 3.84× compared to the best baseline while maintaining generation quality.
By Kaiyuan Deng, Yuchen Li, Gen Li, Yang Xiao, Geng Yuan, Xiaoyong Yuan, Bo Hui, Xiaolong Ma
The paper introduces genesis steering, a method that derives a steering vector from contrastive examples in a model’s own fine‑tuning data, and proposes using membership inference to build proxy datasets when such data are unavailable. Two theoretical results are presented on optimal displacement to the decision boundary and reduced proxy‑estimation bias. Experiments on T"ulu‑3‑8B and Llama‑3.1‑8B‑Instruct show that safety data from T"ulu‑3‑8B’s training mixture produce stronger steering than external datasets, and that membership‑based proxy construction can achieve competitive performance.
By Tora Bodin, Eric Aubinais, Ilaria Torre, Chih-Hong Cheng
The study evaluates recurrent neural networks (RNNs) for classifying tree species from Sentinel‑2 time‑series data, focusing on distinguishing pure from mixed Norway spruce pixels. RNNs outperform Random Forest and XGBoost, achieving a Macro‑F1 of 0.851. Integrated Gradients analysis identifies key spectral‑temporal patterns—especially after day of year 125 and in band B11—that drive class separation, offering insights into phenological periods useful for mapping mixed forest structures.
By Maximilian Kirsch, Christine Preisach
The paper introduces SAE‑IDF, a data‑driven approach that uses sparse autoencoders to encode symbols into interpretable sparse codes, re‑weights dimensions by rarity, and matches units across heterogeneous symbolic systems. On 12 tasks spanning six symbolic system types, SAE‑IDF outperforms dense LLM baselines (Gemma‑2‑9B and Llama‑3.1‑8B) by up to 45 points, consistently across layers and configurations. Each sparse dimension carries a human‑readable meaning, allowing matches to decompose into named concepts, thereby enabling a scalable, universal symbolic backbone.
By Xiaocong Yang, Xuanyu Meng, Yuji Zhang, Jiawei Han, ChengXiang Zhai
PrismQuant introduces a rotation framework that aligns the leading activation eigenspace with the constant group subspace of asymmetric grouped INT4 quantizers, optimizing the alignment via Ky Fan trace maximization and providing a closed‑form solution. The method employs compact Householder transformations for efficient, gradient‑free construction and applies a predictive range law to relate unaligned activation energy and group size to quantization variation. Experiments on large language models up to 70B parameters show PrismQuant achieving state‑of‑the‑art perplexity and accuracy, while delivering significant speedups and memory reductions in deployment scenarios.
By Yanlong Chen, Yining Chen, Song Zhang, Amirhossein Habibian, Yawei Li
arXiv:2610. 14567v2 Announce Type: replace-cross Abstract: This paper develops strong Gaussian approximations (KMT type) for constant learning rate stochastic gradient descent (SGD).
By Joseph Keslin, Wei Biao Wu
The paper demonstrates that pre‑trained language models can perform record linkage and demographic inference from names without task‑specific training. EnsembleLink, an algorithm that combines pre‑trained embeddings with string‑based scorers, achieves performance comparable to or better than prior methods across diverse scripts and formats. Large language models reach up to 83.8% accuracy on U.S. voter files for ethnicity inference, surpassing name‑supplemented BISG and enabling inference for non‑U.S. ethnic categories while reducing bias in geographic racial gap estimates.
By Noah Dasanaike
DecisionRobust is a new benchmark that evaluates the robustness of JEV-style decision models to adversarial attacks across both text and vision inputs. It includes a variety of attack types—semantic text perturbations, image perturbations, and prompt injections—and uses external validity checks and cross-model candidate selection to curate evaluation sets. Experiments on 30 text and 17 vision model configurations reveal that vulnerabilities can transfer across models, robustness varies by task and attack, and label-consistency checks significantly reduce apparent attack success.
By Yifan Lan, Zhongye Liu, Yurui Chang, Hanyu Wang, Yongkang Du, Lu Lin, Jinghui Chen
The paper introduces CAFE, a method that confines Centered Kernel Alignment (CKA) feature distillation to language‑routed adapters in compact multilingual ASR models. By restricting the CKA gradient to adapters that English bypasses, CAFE mitigates the trade‑off where CKA improves non‑English WER at the cost of English performance. On the Open ASR Leaderboard, CAFE reduces average WER from 8.55% to 8.02% and outperforms an adapter‑only capacity control, achieving competitive results across 48 test sets in 24 languages with 254M parameters.
By Tianchi Liu, Nicholas Moratelli, Khurram Azeem Hashmi, Mohammadreza Zolfaghari, Rishabh Jain, Louis Lu, Changdae Park, Amril Nazir
GraphRAG-LTM is a Layered Threat Modeling framework designed to evaluate security vulnerabilities in Graph Retrieval-Augmented Generation (GraphRAG). It introduces three nested attacker access levels—BB Query, WB Query, and Graph-aware WB—that manipulate query ranking, retriever gradients, and graph evidence to degrade answer quality. Experiments across datasets, retrievers, and LLM backbones show that rank feedback alone lowers answer quality, gradient access amplifies query attacks, and deleting bridge triplets further disrupts multi‑hop evidence propagation.
By Yining Tang, Junyu Zhang, Haomiao Qiu, Yongchao Liu, Jiajun Zheng, Zhao Wang, Ziyue Qiao, Chao Wu
The paper "Moral Combat: Exploiting Moral Foundations Theory to Jailbreak LLMs" presents a multi‑agent framework that uses Moral Foundations Theory to reframe harmful prompts in a way that aligns with different moral foundations, thereby enabling large language models to bypass safety guardrails. The authors demonstrate that this approach achieves high single‑trial jailbreak rates across several open‑source and proprietary models, with success increasing under repeated attempts. Their analysis also reveals that existing safety benchmarks are unevenly distributed across moral foundations, focusing on those foundations where models already show biases.
By Adel Khorramrouz, Yi-Chia Wang, Sharon Levy
The paper introduces Length‑Conditioned Majority Voting (LCMV), a test‑time training method that enhances the standard Majority Voting (MV) technique for Large Language Models (LLMs). LCMV incorporates an empirical bias that longer Chain‑of‑Thoughts (CoTs) are less likely to be correct by modeling a length‑conditioned prior with a logistic function. This prior is adapted on unlabeled test data without fine‑tuning the LLM, and the resulting weighted votes consistently outperform standard MV and other length‑ or confidence‑based aggregation strategies across multiple reasoning benchmarks.
By Fiorenzo Parascandolo, Enver Sangineto, Xiaoyi Yu, Wenhui Tan, Jipei He, Ruihua Song, Rita Cucchiara
The paper presents a large‑dimensional analysis of Generalized Total Variation Minimization (GTVMin) for least‑squares federated learning. It shows that Lipschitz observations of the global GTVMin solution concentrate in high dimensions regardless of network structure, and derives exact analytic expressions for performance under both i.i.d. and non‑i.i.d. Gaussian mixture models. These results quantify how regularization balances local adaptation against collaboration, and are validated by experiments on synthetic and real‑world data.
By Ekkehard Schnoor, Malik Tiomoko, Alexander Jung
The paper introduces the Lattice Representation Hypothesis (LRH) to explain why different alignment criteria—such as neighbor overlap, spanning trees, and distances—yield varying views on model representation convergence. It shows that the Platonic Representation Hypothesis (PRH) reflects agreement at the top of a lattice, while evidence comes from criteria near the bottom, and that the Aristotelian Representation Hypothesis (ARH) limits convergence to local neighborhoods due to calibration effects. Experiments on numerous pretrained vision and language models, along with theoretical analyses of linear autoencoders and contrastive Gibbs models, support the LRH’s claim that training procedures and objectives determine which criteria are reliably met across runs.
By Dai Shi, Andi Han, Xiaoyu Li, Luke Thompson, Jos\'e Miguel Hern\'andez-Lobato
ReAl: Refine neural operators via aligning dynamical flow introduces a model‑agnostic plug‑in that enhances pretrained PDE surrogates by adding a refinement stage with a conditional flow‑matching model to recover missing high‑frequency structures, and an alignment stage that generates multiple trajectories and uses trajectory‑level rewards to stabilize long‑horizon predictions. The method improves high‑frequency fidelity, forecasting accuracy, and long‑horizon stability across diverse surrogate architectures and PDE tasks.
By Jing Xu, Zenglin Xu, Li Yi, Ruoxi Jiang, Yuan Qi
The paper investigates the role of first‑moment momentum in reinforcement learning with verifiable rewards (RLVR). Using a bandit formulation with bias‑corrected exponential moving average (EMA) momentum, the authors prove that, when policy gradients are exact, increasing the EMA coefficient β within a safe range accelerates optimization. In practice, they show that maintaining a larger β preserves a non‑vanishing expected gain even as the learning rate shrinks, and experiments on Qwen models confirm that higher β values improve validation accuracy by up to about 2 percentage points.
By Kaifei Wang, Zikang Shan, Liwei Wang, Yinyu Ye, Han Zhong