arXiv AI
Jun 18
Dual Dimensionality for Local and Global Attention
arXiv:2606. 18587v1 Announce Type: cross Abstract: Decoder-only Transformers compute attention over the KV cache of preceding tokens.
Inference efficiency
Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.
arXiv Machine Learning
Jun 18
DiPOD: Diffusion Policy Optimization without Drifting Apart
arXiv:2606. 13795v2 Announce Type: replace Abstract: RL post-training has become increasingly pivotal for improving diffusion policies, but existing diffusion policy-gradient methods are often unstable and cannot achieve reliable policy improvement.
arXiv AI
Jun 18
MIDS: Detecting Stealthy Masquerade and Tampering Attacks on CAN Bus via Bidirectional Mamba
arXiv:2606. 18599v1 Announce Type: cross Abstract: The Controller Area Network (CAN) protocol is the primary communication standard for Electronic Control Units (ECUs) in modern vehicles, but its lack of encryption and authentication exposes it to a range of security threats.
arXiv Machine Learning
Jun 18
Effects of sparsity and superposition on loss in simple autoencoders
arXiv:2606. 18538v1 Announce Type: new Abstract: One of the major difficulties in the mechanistic interpretability of neural networks is the occurrence of polysemanticity, which suggests that each neuron is typically responsible for multiple different tasks, impeding a clean interpretation of their function.
arXiv Machine Learning
Jun 18
Robust and Interpretable Adaptation of Equivariant Materials Foundation Models via Sparsity-promoting Fine-tuning
arXiv:2606. 18691v1 Announce Type: new Abstract: Pre-trained materials foundation models, or machine learning interatomic potentials, leverage general physicochemical knowledge to effectively approximate potential energy surfaces.
arXiv Machine Learning
Jun 18
EfficientRollout: System-Aware Self-Speculative Decoding for RL Rollouts
arXiv:2606. 18967v1 Announce Type: new Abstract: Reinforcement learning (RL) has become a representative post-training paradigm for LLMs, enabling strong reasoning and agentic capabilities.
arXiv Machine Learning
Jun 18
TIGER: Inverting Transformer Gradients via Embedding-Subspace Distance Optimization
arXiv:2606. 18312v1 Announce Type: cross Abstract: Federated learning allows multiple clients to jointly train a shared model by sending gradient updates to a central server while keeping raw inputs local.
arXiv Machine Learning
Jun 18
Reliable Neural-Codec Text-to-Speech by ASR Self-Verification and Distillation: Near-Zero Catastrophic Failures Across Models and Codecs
arXiv:2606. 18323v1 Announce Type: cross Abstract: Open autoregressive neural-codec text-to-speech (TTS) models sound excellent on typical inputs yet suffer stochastic catastrophic failures: on a meaningful fraction of utterances they emit silence, terminate early, or collapse into repetitive or hallucinated content.
arXiv Machine Learning
Jun 18
Ultrafast On-chip Online Learning via Spline Locality in Kolmogorov-Arnold Networks
arXiv:2602. 02056v3 Announce Type: replace-cross Abstract: Ultrafast online learning is essential for high-frequency systems, such as controls for quantum computing and nuclear fusion, where adaptation must occur on sub-microsecond timescales.
arXiv AI
Jun 18
scGTN: Deep Siamese Graph Transformer Network for Single-cell RNA Sequencing Clustering
arXiv:2606. 18672v1 Announce Type: cross Abstract: Single-cell RNA sequencing (scRNA-seq) serves a pivotal role in characterizing gene expression at the cellular level, enabling the identification of cell types and advancing the understanding of cellular heterogeneity.
arXiv Machine Learning
Jun 18
TimeLAVA: Learning-Agnostic Data Valuation for Time Series
arXiv:2606. 18729v1 Announce Type: cross Abstract: Data valuation quantifies the intrinsic quality of individual samples to enable principled data curation, quality control, and robust learning.
arXiv Machine Learning
Jun 18
FORGE: Foundational Optimization Representations from Graph Embeddings
arXiv:2508. 20330v5 Announce Type: replace Abstract: Combinatorial optimization problems are ubiquitous in science and engineering.
arXiv Machine Learning
Jun 18
Complementary Attention Head Pruning for Efficient Transformers
arXiv:2606. 19150v1 Announce Type: new Abstract: The remarkable success of Transformer-based models in natural language processing stems from architectural scaling, which leads to a large number of parameters and hinders deployment in resource-constrained environments.
arXiv AI
Jun 18
Continuous Audio Thinking for Large Audio Language Models
arXiv:2606. 18273v1 Announce Type: cross Abstract: Large audio language models (LALMs) have shown impressive capabilities on diverse audio understanding tasks, ranging from speech transcription to music analysis.
arXiv Machine Learning
Jun 18
Self-attention-based non-linear basis transformations for compact latent space modelling of dynamic optical fibre transmission matrices
arXiv:2406. 07775v2 Announce Type: replace Abstract: Multimode optical fibres are hair-thin strands of glass that efficiently transport light.
arXiv AI
Jun 18
Generalized Kullback-Leibler Divergence Loss
arXiv:2503. 08038v2 Announce Type: replace-cross Abstract: In this paper, we delve deeper into the Kullback-Leibler (KL) Divergence loss and mathematically prove that it is equivalent to the Decoupled Kullback-Leibler (DKL) Divergence loss that consists of (1) a weighted Mean Square Error (wMSE) loss and (2) a Cross-Entropy loss incorporating soft labels.
Hugging Face Trending Papers
Jun 17
Formal Verification of Learned Multi-Agent Communication Policies via Decision Tree Distillation
Multi-agent reinforcement learning (MARL) enables agents to develop coordination strategies through emergent communication, but neural policies lack the formal safety guarantees required for safety-critical robotic deployment in drone swarms and autonomous vehicle fleets. We present the first end-to-end framework for safety verification of learned multi-agent communication policies through policy abstraction: neural policies are distilled into interpretable decision trees, then formally verified, with empirical validation confirming that verified safety properties transfer to original networks.
Hugging Face Trending Papers
Jun 17
Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation
Post-training of reasoning language models is commonly driven by supervised distillation and reinforcement learning with verifiable rewards. Distillation often relies on chain-of-thought annotations that are expensive to obtain and may themselves be noisy, incomplete, or partially incorrect; even when the final solution is correct, an imperfect rationale can interfere with learning.
Hugging Face Trending Papers
Jun 17
Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning
Unified multimodal models (UMMs) interleave generated ''visual thoughts'' (VTs) with text reasoning to improve spatial tasks. This incurs roughly an order-of-magnitude inference cost from multi-step diffusion.
Hugging Face Trending Papers
Jun 17
FOSC-X: An Extended Framework for Optimal Local Cuts and Non-Horizontal Cluster Selection from Clustering Hierarchies
Extracting a flat clustering solution from a hierarchy is a common task in practical cluster analysis and can be formulated as an optimisation problem. Existing approaches focus on finding a single optimal solution.