The paper studies the symmetry properties of the differential neural tangent kernel (DNTK) for overparameterized physics‑informed neural networks. It shows that the infinite‑width DNTK is invariant under the orthogonal group for both two‑layer and deep networks, regardless of activation or bias. The authors further prove that integral operators induced by group‑orbit operators share spectra and that the fixed DNTK exhibits zonal symmetry on the unit sphere, enabling derivation of eigenvalue decay rates for shallow RePU networks with zero bias.
By Bangti Jin, Longjun Wu
The paper introduces concept bottlenecks into deep graph generators to enhance interpretability and steerability in molecular graph generation. By embedding human-interpretable concepts, the authors demonstrate improved distributional fidelity (lower FCD) while maintaining validity and uniqueness. They also present a Concept Controller optimization method for steering generation, evaluated within a one-shot molecular graph generation framework.
By Sarabeshwar Balaji, Ritesh Kumar, Lav R. Varshney
The paper investigates how language‑model agents use external procedural skills and discovers a bias called Procedural Exposure Bias (PEB), where repeated exposure to a skill’s pattern makes the model favor its associated action even when the context is unchanged. PEB is much weaker when the skill is presented visually rather than textually, mainly due to visual encoding rather than layout. Experiments across multiple environments show that visual skills lead to higher task success than textual ones, highlighting the importance of skill representation for agent decision‑making.
By Qing Yang, Yuqi Xiong, Pengcheng Huang, Zhipeng Xu, Zhenghao Liu, Zulong Chen, Yu Gu, Ge Yu
SpeeDiff is a latent diffusion model that jointly trains a VAE and diffusion network from scratch, using a Tweedie Pixel Reconstruction loss to prevent latent collapse. The approach allows scaling a transformer‑based architecture and improves representation alignment. Experiments show the SpeeDiff‑XL model trains 140× faster than Vanilla SiT and 61× faster than REPA, achieving an FID of 1.50 on ImageNet 256×256 and 1.53 on 512×512 without guidance.
By Bingliang Zhang, Wenda Chu, Yizhuo Li, Linjie Yang, Yisong Yue, Katherine L. Bouman, Yang Song, Qiushan Guo
MVRCap is a new video captioning framework that treats caption generation as a reinforcement learning problem guided by structured verification. It introduces a Fine‑Grained Verifiable Reward system built on three pillars—Hierarchical Concept Coverage, Factuality Verification, and Conciseness Constraint—supported by the newly created MVRCap‑QA and MVRCap‑SFT datasets. Experiments show that MVRCap outperforms existing methods on benchmarks such as VDC, CAPability, and VC4VG.
By Weihuang Lin, Xianing Chen, Zhibin Wang, Congzhi Zhang, Yiwei Ma, Jiayi Ji, Miao Chen, Jun Song, YuCheng, Bo Zheng, Xiaoshuai Sun, Rongrong Ji
The paper investigates how to detect money‑laundering patterns across separate bank transaction graphs using federated learning. By leveraging transactions that both banks already record, the authors approximate remote account states locally and run multi‑layer Graph Neural Networks without exchanging sensitive account data. Experiments on AMLworld HI‑Small show that a simple baseline using locally available information yields the greatest performance, while adding limited context such as transaction counts improves results; however, sharing detailed remote states requires excessive noise that diminishes utility.
By Lidia Losavio, Dario Fenoglio, Martin Gjoreski, Marc Langheinrich
The paper evaluates Variational Autoencoders, Generative Adversarial Networks, and Diffusion Models for unsupervised anomaly detection in federated predictive maintenance. It compares full and partial federation setups, showing that full federation improves GAN performance but VAE and DDPM models are more robust. Decoding‑only sharing in Diffusion Models achieves similar utility to full federation while cutting communication costs.
By Usevalad Milasheuski, Piero Baraldi, Enrico Zio, Stefano Savazzi
The paper introduces PropMe, a framework that evaluates both the capability of large language models (LLMs) to reproduce training data via prefix attacks and their natural propensity to do so without adversarial prompting. Using SimpleTrace to compare every generation against the full indexed training corpus, the authors test models such as Comma, DFM Decoder, and OLMo3 32B across datasets like Common Pile, Dynaword, and Dolma3. Results show that natural sentence prompts produce less training text than prefix attacks, but unprompted or single-word prompts can reproduce as much or more, often involving duplicated licenses, code, and templates; continual pre‑training only modestly reduces verbatim reproduction.
By Gianluca Barmina, Peter Schneider-Kamp, Lukas Galke Poech
TextCloak is a reinforcement‑learning driven framework that generates unlearnable examples (UEs) to protect textual data from unauthorized fine‑tuning of large language models (LLMs). It transforms clean text into perturbations that preserve meaning and naturalness while degrading the utility of models trained on them. Experiments on six datasets and nine LLMs show that TextCloak consistently impairs unauthorized fine‑tuning and remains robust across different architectures, training settings, and adaptive attacks.
By Chengshuai Zhao, Pingchuan Ma, Dawei Li, Bohan Jiang, Zhiyuan Yu, Zhen Tan, Huan Liu
The paper presents a Gaussian approximation for the privacy loss distribution in differentially private stochastic gradient descent (DP‑SGD) when subsampling is used. It shows that, after truncating rare large losses, the privacy loss behaves like a Gaussian with the same mean and variance, up to a small shift that grows only logarithmically with the error tolerance. This leads to tighter variance‑based bounds for subsampled mechanisms, improving upon previous Renyi‑divergence analyses and explaining the effectiveness of the connect‑the‑dots method for random allocation sampling.
By Vitaly Feldman, Moshe Shenfeld
Geodesic‑SVDD introduces a distributed learning framework for structured tabular data that transforms each client’s task into a one‑class density estimation on a hypersphere, enabling aggregation of local models as probabilistic manifolds in a single communication round. The method employs a Residual Spherical Encoder and a Geodesic Latent Mixup to enforce strict hyperspherical constraints, preventing manifold collapse, and proposes a One‑Shot Product‑of‑Experts protocol for privacy‑preserving aggregation without iterative synchronization. Experiments on five tabular benchmarks and varying label skew levels show that Geodesic‑SVDD achieves the highest mean AUROC among seven federated methods, surpasses federated tree ensembles by up to 21 AUROC points, and its One‑Shot variant matches centralized Deep SVDD under IID and mild skew conditions.
By Andrea Moleri, Christian Intern\`o, Markus Olhofer, Barbara Hammer, Ali Raza
The paper establishes identifiability results for constant‑width residual networks that use polynomial activations and bias terms. For activations of degree at least five with generic coefficients, the mapping from parameters to network outputs is generically one‑to‑one. For activations of degree at least three that lack a linear term after a suitable input translation, the generic fiber consists of translations of biases combined with scaling of weights and biases by real roots of unity, and these findings also apply to bias‑free networks. Additionally, the authors derive necessary conditions for critical points of the ResNet parametrization, which are important in singular learning theory.
By Isabel Dahlgren, Giovanni Luca Marchetti, Kathl\'en Kohn
The paper introduces an error‑diffusion (ED) optimizer that replaces the full‑precision latent copy of binary‑weight language models with a small 2‑8‑bit residual buffer, reducing training state per weight from 97 bits to 3‑9 bits—up to 19× less. ED accumulates unrealized weight updates in the buffer and flips the binary weight when the accumulated value crosses a threshold aligned with the quantization grid, enabling efficient training on large transformers with minimal loss compared to latent‑weight baselines. The method scales to models with up to 19 M parameters on enwik8 and 70 M on wikitext‑103, maintaining performance within a few percent of reference while allowing on‑device adaptation within a 275 MB memory budget.
By Pierre-Olivier Bonin
The paper investigates the space complexity of attention mechanisms when key/query cosine similarities are constrained to a small interval around zero, reflecting the anisotropy observed in real transformers. It presents a nearly space‑optimal algorithm alongside a matching lower bound, showing that the distribution of similarities in the lower bound aligns with empirical anisotropy. The study also demonstrates that a simple uniform sampling approach is nearly optimal for typical parameter ranges, explaining its effectiveness on real datasets.
By Justin Y. Chen, Ying Feng, Piotr Indyk, Michael Kapralov, Ekaterina Kochetkova, Boris Prokhorov
The paper introduces a JiT-based denoising defense for radio signal automatic modulation classification, employing a 1D Transformer trained via flow matching to map adversarial examples onto a clean signal manifold. Experiments on the RML2018.01A dataset show the method outperforms PGD-based adversarial training, improving robust accuracy by at least 20.58% under PGD attacks and clean accuracy by 10.20%. It also demonstrates stronger defense against unseen attack types, achieving up to a 39.72% improvement in robust accuracy at higher attack strengths.
By Shiwei Liu, Linyuan Wang, Nuolin Sun, Libin Hou, Bin Yan
The thesis investigates practical optimization techniques used in modern machine learning, such as submodel training, gradient clipping, and layer-wise non‑Euclidean directions, by analyzing their actual objectives, assumptions, updates, and computations rather than simplified proxies. It demonstrates that Independent Subnetwork Training can converge to a shifted fixed point due to sketching bias, while Model‑Agnostic Sparsified Training incorporates the sketch distribution to achieve unbiased gradients and convergence guarantees. The work also explores private optimization, showing that adaptive quantile clipping retains bias unless a suitable schedule is used, and combines smoothed normalization with error compensation to obtain distributed private convergence without bounded gradient assumptions. Additionally, it develops a layer‑wise generalized smoothness model for optimizers based on linear minimization oracles, validates it through the Gluon framework, and highlights the importance of precise orthogonalization in Muon‑style methods.
By Egor Shulgin
The study reports the first computational design of de novo miniprotein binders that target the IL‑23 cytokine subunit p19, specifically the IL‑23R‑binding face. Using a length‑robust interface metric (ipSAE) derived from AlphaFold‑predicted aligned error, the authors identified 42 of 50 designs with high interface quality, and five all‑alpha‑helical leads that engage all six targeted hotspots without affecting the p40 subunit. These candidates are proposed as potential neutralizers of IL‑23 activity, offering a smaller, potentially cheaper alternative to existing anti‑p19 antibodies.
whyItMatters":"The work introduces a novel computational approach to generate miniprotein candidates that could inhibit IL‑23‑mediated inflammation, addressing a therapeutic need unmet by current large antibody treatments."
By Raghav Paladugu, Fardina Fathmiul Alam
arXiv:2610. 36864v1 Announce Type: new Abstract: Mutual information (MI) is fundamental to representation-learning analysis in deep networks, but in high dimensions MI estimators depend on hyperparameters whose bias often dominates the signal.
By Johan Mylius-Kroken, Elisabeth Wetzer, Ali Ramezani-Kebrya, Robert Jenssen, Kristoffer Knutsen Wickstr{\o}m
INTR++ is an Interpretable Transformer that builds on Prompt-CAM by adding a lightweight Prompt Adapter Module (PAM) to make prompts adaptive to domain-specific image characteristics. This design allows the model to maintain consistent interpretability and performance across diverse domains such as medical imaging, microscopic imaging, remote sensing, and object-centric data, while adding only a few extra parameters. The authors demonstrate that INTR++ improves accuracy, interpretability, and adaptability compared to previous Interpretable Transformers.
By Dipanjyoti Paul, Arpita Chowdhury, Maneet Chatterjee, Daichi Kitaguchi, Tanya Berger-Wolf, Tetsuya Sakurai, Wei-Lun Chao
The paper introduces SimpleLoRIF, a scalable gradient‑embedding technique that pinpoints fine‑tuning examples whose gradients align with unsafe behavior in large language models. By applying this method, the authors create SafeTulu, a version of Tulu 3 with the 20 % of most unsafe‑aligned examples removed, which reduces GCG attack success by up to 22 % across several LLMs while largely preserving or improving capability. The study also highlights previously under‑studied safety‑degrading data sources such as tool‑execution failures and narrative arithmetic.
By Saket Reddy, Amro Abbas, Anshuman Suri, Ari Morcos, David Schwab, Pratyush Maini