PointLAM introduces a new point-based 3D object detection architecture that addresses efficiency and fidelity trade-offs inherent in LiDAR point cloud processing. It employs a Laplacian Point Sampler (LPS) to accelerate downsampling while preserving foreground structure, and a Local Hadamard Aggregator (LHA) that replaces costly continuous interactions with a topology‑aware gating mechanism. Combined with Bi‑Directional Mamba layers, the resulting Local Attentive Mamba (LAM) block delivers competitive performance on nuScenes and Waymo datasets, outperforming voxel‑based competitors in detecting small objects and handling extreme sparsity with a smaller computational footprint.
By Xuanming Shang, Weijia Zhang, Chao Ma
ECG-Mamba-V2 refines a bidirectional Vision Mamba encoder for multi‑label 12‑lead ECG classification by appending the class token at the sequence end, summing forward and backward outputs without ½ scaling, and applying uniform dropout across blocks. These architectural tweaks yield a 0.6494 macro AUPRC and 0.9716 macro AUROC on the PhysioNet/CinC Challenge 2021, outperforming its predecessor while using 34% fewer parameters and achieving 38% higher throughput.
By Huawei Jiang, Husna Mutahira, Shibo Wei, Gan Huang, Vladimir Shin, Dongryeol Ryu, Juneho Yi, Mannan Saeed Muhammad
The paper introduces a general framework for multi-domain clustering using measure quantization, where a shared set of cluster prototypes is learned by minimizing a probability metric (e.g., Sinkhorn divergence or Maximum Mean Discrepancy) between each domain’s probability measure and the prototype measure. Data points are assigned to clusters either by nearest centroid or via optimal transport, a collaborative strategy that couples all samples within a domain. A mini‑batch optimization strategy makes both fitting and assignment scalable, reducing memory and computational cost while preserving clustering performance, and experimental results on five multi‑domain benchmarks (image, audio, and sensor data) show that the Sinkhorn‑based method consistently outperforms classical and multi‑domain clustering baselines, even when scaling to hundreds of thousands of samples.
By Rafael Pereira Eufrazio, Eduardo Fernandes Montesuma, Charles Casimiro Cavalcante
arXiv:2609.21432v1 Announce Type: new
Abstract: Post-training plays a pivotal role in enhancing the reasoning capabilities and task-specific expertise of large language models (LLMs). Despite recent...
By Kaichen Zhang, Yuzhong Hong, Junwei Bao, Hongfei Jiang, Yang Song, Dingqian Hong, Hui Xiong
FOCAL‑VLA is a framework that improves vision‑language‑action models by combining subtask‑guided geometry distillation with implicit world modeling. It transfers geometric knowledge from VGGT to focus on subtask‑relevant image regions and uses Track4World features to capture future 3D evolution, guiding action generation without running these models at inference time. Experiments demonstrate that FOCAL‑VLA outperforms baselines on both simulation benchmarks and real‑world manipulation tasks.
By Zhiyuan Gao, Di Wen, Yanxiang Zhan, Mohammad Khoshnazar, Jeroen Sch\"afer, Kunyu Peng, Michael Beetz
Draft-OPD introduces an on‑policy distillation method for speculative draft models, addressing the mismatch between supervised fine‑tuning and inference by letting the target model supervise the drafter on draft‑induced states. The approach uses target‑assisted rollouts for stable continuations and replays drafting from error positions exposed during verification, enabling the drafter to learn from both accepted and rejected proposals. Experiments demonstrate that Draft‑OPD achieves more than five‑fold lossless acceleration across diverse tasks, outperforming prior draft models such as EAGLE‑3 and DFlash by 23 % and 13 % respectively.
By Haodi Lei, Yafu Li, Haoran Zhang, Shunkai Zhang, Qianjia Cheng, Xiaoye Qu, Ganqu Cui, Bowen Zhou, Ning Ding, Yun Luo, Yu Cheng
LiteMedCoT-VL is a parameter‑efficient pipeline that transfers chain‑of‑thought reasoning from a 235B teacher model to a 2B student model using LoRA fine‑tuning on explanation‑enriched data. The approach enables a compact vision‑language model to perform medical visual question answering without relying on image captions, achieving 64.9% accuracy on the PMC‑VQA benchmark—an 11‑point improvement over the zero‑shot Qwen3‑VL‑4B baseline. Visual grounding analysis confirms that the model bases its predictions on image content rather than textual priors.
By Runze Ma, Shunbo Jia, Haonan Lyu, Guo Liu, Caizhi Liao
The paper investigates how on-policy self‑distillation can alter a model’s behavior by conditioning on privileged information. It contrasts attractive self‑distillation, which pulls a model toward a privileged teacher, with repulsive self‑distillation, which pushes it away, showing that attraction reduces exploratory reasoning while repulsion lengthens responses and can destabilize the model. The authors propose a contrastive self‑distillation objective that combines attraction to a correct‑solution teacher with repulsion from an incorrect‑solution teacher, finding that this approach improves reasoning performance across various model types while keeping response lengths stable.
By Anton Baumann, Akmal Ashirmatov, Leo Schmidt-Traub, Frederike L\"ubeck, Jonas H\"ubotter, Thomas Kleine Buening, Andreas Krause
The paper presents two optimizations for forced alignment of audio to text using the Viterbi algorithm. The first optimization applies the Hirschberg algorithm to reduce memory usage from 140 GB to 5 MB for three‑hour inputs and speeds up alignment to one‑third the time of torchaudio on a CPU. The second optimization models alignment as a constrained random walk, enabling pruning that yields an additional 2× speedup on inputs longer than 20 minutes while maintaining over 98 % alignment accuracy.
By Lawry Sorenson, Michael Crandall, Eric K. Ringger, Stephen D. Richardson
Elastic Threshold Attention (ETA) is a trainable attention mechanism that dynamically predicts contextual thresholds from query representations, enabling selective pruning of KV cache tokens during long‑context decoding. By multiplicatively suppressing sub‑threshold logits during training, ETA avoids representation collapse and eliminates localized attention sinks, allowing a 1.45B model to match dense attention performance at roughly 85% training sparsity and 38% active decode density. At inference, a custom Triton kernel achieves up to 2.5× faster decoding on sequences up to 512K tokens, and an offline calibration step can further reduce compute by 27% by freezing per‑head thresholds.
By Themistoklis Haris, Henry Li, Maryam Karimzadehgan
The paper introduces Probability‑Flow Distillation (PFD), a new method for matching parameter distributions in diffusion‑based models. It extends the particle variational inference framework of Variational Score Distillation to Score Distillation Sampling (SDS) and Score Distillation via Inversion (SDI), revealing that SDS focuses on mode collapse while SDI converges to a contracted distribution. By replacing a single Euler step in SDI with a full reverse probability‑flow ODE solve and simplifying the gradient, PFD achieves distribution matching with only a forward ODE solve, and experiments on synthetic data, CelebA, and text‑to‑3D tasks confirm its effectiveness.
By Rohith Ramanan, A. N. Rajagopalan
VISPATH is a visual‑intent‑guided path reasoning framework designed for multimodal knowledge graph question answering (MM‑KGQA). It first identifies a reliable starting entity by fusing multimodal grounding with graph‑structural cues, then iteratively discovers and refines reasoning paths using hop‑specific multimodal intent and a reasoning‑chain pruning step. The framework is evaluated on the newly introduced VISPATH‑Bench, which tests two‑to‑four‑hop reasoning, and demonstrates consistent improvements over strong baselines, even surpassing GPT‑5.4 when using GPT‑4o as the backbone.
By Jinke Wu, Zhengpin Li, Mengzhe Jia, Yang Li, Wentao Zhang
ZYT-World is a real‑time, controllable world model designed for closed‑loop autonomous‑driving simulation. It generates a full fisheye‑pinhole camera rig at native resolution, using Plucker adapters for camera geometry, ego‑motion adaptive layer normalization for motion control, and a lightweight pixel‑aligned layout to condition traffic participants. The model achieves high fidelity to a 40‑step teacher while being over 100 times faster, and includes an implicit‑memory module that preserves place‑specific evidence for long‑horizon stability.
By Boni Hu, Xiong Wei, Haoming Huang, Yong Huang, Chenbo Wang, Yi Yang, Jiancheng Wang, Ruicheng Zhu, Zhimin Yang, Guanglai Liu, Qiaowan Jin, Dongzhuo Wang, Haiwei Kuang, Jiajun Fan, Yue Wu, Jiaxin Wei, Hao Sun, Feihong Yan, Wei Bi, Kaixuan Wang, Zichao Guo, Xiaozhi Chen
RheoSampling tackles the one‑hot dilemma in stochastic dynamic‑tree speculative decoding by decoupling token sampling from tree construction and verification. It injects a sampled token into deterministic top‑K slots, assigning distinct proxy probabilities for expansion and pruning while preserving the true sampling probability for verification, thereby achieving lossless, context‑aware top‑K construction with stochastic sampling. Experiments on various LLMs show higher acceptance rates and speedups compared to existing dynamic‑tree methods.
By Qiao Hu, Yepeng Weng, Bo Zhang, Takehisa Yairi
arXiv:2609.21619v1 Announce Type: new
Abstract: On-policy distillation (OPD) improves reasoning models by learning the token-level discrepancy between a stronger teacher and an on-policy student. How...
By Qiangqiang He, Jin Li, MingCai Chen
arXiv:2609.21172v1 Announce Type: new
Abstract: Large language models (LLMs) are moving onto mobile devices for increasingly diverse workloads over text, images, video, and audio. These applications...
By Zhihao Shu, Md Musfiqur Rahman Sanim, Jie Hu, Kun Yuan, Minghai Qin, Gagan Agrawal, Wei Niu
arXiv:2609.21450v1 Announce Type: new
Abstract: Post-training weight-activation quantization reduces the memory and inference costs of large language models, but aggressive W4A4 quantization remains...
By Yamato Narita, Issei Sato
arXiv:2601.11719v4 Announce Type: replace
Abstract: Self-supervised learning, in the context of foundation model training, is a powerful pre-training method for learning feature representations witho...
By Ho Fung Tsoi, Dylan Rankin
arXiv:2603.29715v2 Announce Type: replace
Abstract: Nonnegative matrix factorization (NMF) approximates a nonnegative matrix, X, by the product of two nonnegative factors, WH, where W has r columns a...
By Giovanni Seraghiti, K\'evin Dubrulle, Arnaud Vandaele, Nicolas Gillis
arXiv:2505.10202v2 Announce Type: replace
Abstract: Large Language Models (LLMs) have achieved remarkable success but face significant computational and memory challenges, particularly due to their e...
By Jintian Shao, Hongyi Huang, Jiayi Wu, YiMing Cheng, ZhiYu Wu, You Shan, MingKai Zheng