The paper introduces DS-MoE, a framework that selects a compact subset of experts in Mixture-of-Experts models using difference-of-submodular (DS) optimization. It analyzes the loss degradation via a second-order Taylor expansion to uncover redundancy and synergy among experts, and formulates the selection objective to decouple these effects. A majorization-minimization algorithm with monotonicity guarantees efficiently identifies the optimal subset, and experiments show DS-MoE outperforms state‑of‑the‑art baselines.
By Zheng Lin, Shaoke Fang, Yuxin Zhang, Jinfeng Xu, Zihan Fang, Zhe Chen, Wei Ni, Jun Luo, Symeon Chatzinotas
EP-Flow introduces a new framework for predicting disordered crystal structures without requiring site-level disorder annotations. It uses an Occupancy Distribution Matrix (ODM) to represent continuous site-by-species occupancies and enforces constraints through a transportation polytope. The method jointly generates occupancies, fractional coordinates, and lattice parameters, achieving state‑of‑the‑art performance on formula‑conditioned disordered CSP benchmarks and recovering chemically meaningful local disorder patterns.
By Qiuliang Liu, Liming Wu, Qi Li, Zhonglong Peng, Chang Chen, Xiaolong Chen, Wenbing Huang, Shifeng Jin
arXiv:2610. 02191v1 Announce Type: new Abstract: While Large Language Models (LLMs) have demonstrated striking capabilities on frontier mathematical problems, it remains unclear whether they possess the structural mathematical understanding underlying their solutions.
By Shuo Xing, Zilin Dai, Chengyuan Qian, Fangzhou Lin, Wenjing Chen, Ping He, Pan Lu, Alvaro Velasquez, Mohit Bansal, Zhengzhong Tu
The paper examines how large language models (LLMs) express uncertainty compared to humans, noting that humans use verbal markers like "possible" or "likely" to convey metacognitive awareness. By curating a corpus of human uncertainty markers and benchmarking LLMs against it, the authors find that LLMs encode these markers with numerical levels that differ substantially from human usage. They introduce METHODNAME, an optimization-based algorithm that learns an optimal uncertainty profile over verbal markers directly from LLM outputs, enabling a direct comparison of confidence semantics and revealing systematic disparities in verbal expressions.
By Jinhao Duan, Zicheng Liu, Zijie Liu, Kaidi Xu, Tianlong Chen
arXiv:2610. 00545v1 Announce Type: new Abstract: We study adversarial online maximization of nonnegative, non-monotone DR-submodular functions over compact convex down-closed sets.
By Vaneet Aggarwal
The paper introduces attention kernels that replace the exponential function in transformer softmax to better handle operator learning on probability measures with heavy-tailed (polynomial) distributions. Two new benchmarks with closed‑form targets are constructed to evaluate how different kernel growth rates and data preprocessing affect performance. The study finds that slower‑growing kernels prevent ensemble collapse on heavy‑tailed tasks, while softmax with symlog preprocessing only succeeds on a subset of problems, and that all kernels perform similarly on Gaussian data.
By Kailen Hargenrader, Edoardo Calvello, Bohan Chen
MoRA is a framework for pruning Mixture-of-Experts (MoE) models by learning a router bias for each expert and optimizing it with a language‑modeling loss and a routing‑diversity regularizer. The learned biases sharpen routing distributions to identify critical experts and encourage diverse routing preferences. After pruning, MoRA uses an expert approximation mechanism that approximates the outputs of pruned experts with affine transformations of remaining experts, further improving performance.
By Yushuai Sun, Zikun Zhou, Lin Gao, Jun Yu, Wenjie Pei
WOMBAT is a benchmark comprising 14 whitebox graph neural networks (GNNs) whose message‑passing weights are manually set to detect specific SMARTS motifs. Each model’s decision rule is explicitly known, providing a ground truth for attribution that allows researchers to identify and study errors in post‑hoc explainers such as GNNExplainer, PGExplainer, and Integrated Gradients. The authors validate the models on millions of PubChem molecules, demonstrate how Integrated Gradients can be misled to spread attribution, and release the dataset, models, and evaluation code for future XAI tool development.
By Dominik Matuszek, Bartosz Zieli\'nski, Tomasz Danel, Dawid Rymarczyk
Autoregressive modelling, successful in language tasks, is applied to mineral‑exploration drillholes where lithology is revealed sequentially from shallow to deep. The authors introduce DrillBench, a benchmark of 49,671 Western Australian drillholes for next‑layer prediction and autoregressive stratigraphic generation across a graded transfer spectrum. Experiments show that spatial conditioning helps locally but fails under strong shift, while autoregressive models transfer more robustly; a backbone‑agnostic recipe combining large‑scale pretraining and spatial retrieval improves generalisation, especially in distant geological splits.
By Yihao Ding, Daniel Yitian Su, Yiran Zhang, Christopher M. Gonzalez, Wei Liu
AF‑Muon is an AdamW‑free extension of the Muon optimizer that retains Muon’s matrix update for hidden weights while applying a support‑aware finite‑cap linear minimization oracle to tied vocabulary tables and an RMS‑normalized update for one‑dimensional auxiliary parameters. This design eliminates second‑moment state, reducing optimizer‑state memory by about 20% compared to Hybrid Muon. Across nine tied‑token settings—including decoder‑only language models, T5‑style encoder‑decoders, and ImageGPT‑style variants—AF‑Muon consistently improves mean validation loss and perplexity over both Hybrid Muon and a SCION‑style Sign endpoint, with robust gains confirmed by long‑horizon runs and hyperparameter studies.
By Arash Lagzian, Paniz Halvachi, Junming Zhang, Zhouhan Lin, Dianbo Liu
The paper investigates how treating the optimization objective as a design variable can improve Remaining Useful Life (RUL) prediction in predictive maintenance. Five model architectures are compared under single‑objective and multi‑objective hyperparameter optimization, with the latter using NSGA‑II and Entropy‑CRITIC weighting to balance accuracy and prediction timeliness. Results on NASA C‑MAPSS and BackBlaze datasets show that multi‑objective optimization reduces directional imbalance in predictions and can alter model rankings, highlighting the importance of objective choice in RUL modeling.
By Tugrul Cabir Hakyemez, Ener Uras Gokhan
The paper introduces Function-Structured Graph Reinforcement Learning (FSG‑RL), a framework that links subproblem graphs to Python code and uses multiple verifiers for feedback. It first trains a policy via supervised fine‑tuning to generate code from function graphs, then refines it with Group Relative Policy Optimization (GRPO) that employs answer‑gated rewards and span‑level credit assignment. On a benchmark combining GSM8K, MathQA, MATH, and Omni‑MATH, GRPO raises final‑answer accuracy from 43.25 % to 67.50 % and full‑solution success from 32.25 % to 52.25 %, with further improvements when teacher supervision is added.
By Zihan Liu, Xurong Xie
The paper tackles the challenge of converting uplink to downlink channel covariance matrices in FDD massive MIMO systems, where learning‑based methods lose accuracy as the antenna count grows. It identifies that the mean angle of arrival creates a phase ramp whose oscillation rate increases with array size, making fixed‑size datasets sparse. The authors propose a deramping technique that estimates and removes this ramp before learning, reducing estimation error across three different learners and maintaining superiority over model‑based benchmarks at large array sizes.
By Melih Can Zerin
STEER is a sampling method for relational foundation models that reduces inference cost by focusing on the most relevant tables for a prediction task. It uses a large language model to rank foreign‑key edges in the database schema into relevance tiers, then assigns traversal probabilities based on these tiers. Evaluated on three state‑of‑the‑art RFMs, STEER cuts inference context size by roughly 40% on average while preserving or improving accuracy.
By Abdalla Mohamed, Ashraf Aboulnaga
The paper introduces Reachability Analysis-Informed Reinforcement Learning (RARL) for designing deterministic multi‑impulse interplanetary transfers. RARL uses local first‑order reachability maps to bound velocity perturbations and selects intermediate waypoints, which are then translated into maneuvers via Lambert reconstruction and a terminal two‑impulse solution. Numerical experiments on an Earth‑Mars benchmark show that RARL achieves a mean maneuver cost only 1.72% above a validated convex programming reference and can be trained once to handle a wide range of departure states, achieving 100% feasibility on 10,000 held‑out Monte Carlo departures.
By Yashdeep Chaudhary, Roberto Armellin, Harry Holt
The paper introduces a ground‑truth framework for disentangling uncertainty into epistemic and aleatoric components using sample‑conditional pointwise posterior risk. It evaluates current methods, finding that Spectral‑normalized Neural Gaussian Processes and Variational Latent Gaussian Processes best recover the ground‑truth uncertainty, while most methods align more closely with posterior variance and miss predictor bias. The study also explores the entanglement of estimated uncertainties and the impact of modeling choices, providing practical guidance and releasing 13 semi‑synthetic datasets for further validation.
By Frieder Wizgall, Georg Tirpitz, Moritz Seiler, Kerstin Ritter, B\'alint Mucs\'anyi
The paper presents a theoretical analysis of symmetric autoencoders, a class of deep learning architectures frequently used in machine learning tasks. It distinguishes between different symmetric designs and shows that the reconstruction error of orthonormal symmetric autoencoders can be interpreted via the Eckart‑Young‑Schmidt theorem. Building on this insight, the authors propose an EYS‑based initialization strategy using repeated SVD, and validate its effectiveness through numerical experiments comparing it to conventional deep autoencoders.
By Simone Brivio, Nicola Rares Franco
The paper argues that evaluating anonymity in synthetic data generation must focus on the generative model rather than just the resulting dataset. It interprets GDPR definitions of personal data and anonymization under realistic model-access scenarios, mapping these to state‑of‑the‑art privacy attacks. The authors conclude that synthetic data alone is insufficient for anonymization, and that Differential Privacy offers stronger protection than Similarity‑based Privacy Metrics.
By Georgi Ganev, Emiliano De Cristofaro
HUANet is a deep neural network architecture that unrolls the Alternating Direction Method of Multipliers (ADMM) into a trainable model for accelerating parametric constrained convex optimization. It embeds a hard‑constrained neural network in each ADMM iteration, using a differentiable correction stage to enforce affine equalities of the primal subproblem. The method also incorporates first‑order optimality conditions into a self‑supervised training loss, and numerical experiments on benchmark problems and a control application demonstrate its effectiveness in speeding up constrained convex optimization.
By Trinh Tran, Binh Nguyen, Truong X. Nghiem
The paper proposes a structured version of the Information Bottleneck (IB) that separates label-relevant structure from within-condition variation using a dual-bottleneck formulation. It introduces a conditional KL term that targets within-condition information, allowing explicit control over nuisance-like variation in learned representations. Experiments demonstrate improved performance in low-data classification and consistent gains on dense prediction tasks.
By Jingyao Zhang, Yuxuan Li, Lu Han, Ali Anaissi, Nguyen H. Tran