REAL-Q introduces a new post‑training quantization approach for large language models that replaces the traditional single closed‑form second‑order solver with a fine‑grained, dynamic block‑wise gradient descent applied after every 128‑column block. By aligning the surrogate loss with the end‑to‑end objective and using a sliding window for smooth cross‑layer transitions, REAL‑Q mitigates error propagation and information misalignment. Experiments on LLaMA‑3.1 and Qwen3 show up to ~49% reduction in end‑to‑end KL divergence compared to state‑of‑the‑art methods.
By Qian Zhang, Yaoming Li, Zhewen Tan, Yanshu Wang, Heng Lu, Kun Su, Zongwei Lv, Wenhan Yu, Yongge Ma, Yinjun Han, Ruikuang Liu, Tong Yang
arXiv:2609.01338v1 Announce Type: cross
Abstract: On-device mobile Large Language Model (LLM) inference is gaining significant attention. However, mobile devices operate in highly dynamic multitaskin...
By Hongseung Yu, Minsung Kim, Jongseok Park, Kyunghan Lee
The paper introduces a method for generating multilingual reasoning traces for medical question answering using large language models. It creates 500,000 reasoning traces in English, Italian, and Spanish by retrieving medical information from Wikipedia and applies them to MedQA and MedMCQA datasets extended into Italian and Spanish. The approach improves performance in both few‑shot in‑context learning and supervised fine‑tuning, achieving state‑of‑the‑art results for 8B‑parameter LLMs and releasing all resources for further research.
By Pietro Ferrazzi, Aitor Soroa, Rodrigo Agerri
The paper presents a Quadratic Constrained Binary Optimization (QCBO) framework that provides provable guarantees for training quantized neural networks. It characterizes the topology of zero‑loss level sets, compiles finite‑depth architectures into bounded QCBOs, and introduces a sample‑wise Decomposed Lower‑Bound Optimization (DLBO) to scale Ising‑based optimization. Experiments on a coherent Ising machine show high accuracy on binary Fashion‑MNIST at 1.1‑bit precision and validate the approach on multi‑class datasets.
By Wenxin Li, Chuan Wang, Hongdong Zhu, Qi Gao, Yin Ma, Hai Wei, Kai Wen
The paper evaluates the effectiveness of tolerance‑based conformance tests for INT8 quantized GEMM kernels used in large language models. By injecting nine faults into a Qwen3‑1.7B reference pipeline, the authors show that most faults shift outputs by at most one bfloat16 spacing, rendering a tolerance of one spacing blind to these errors. They further demonstrate that requantizing weight scales to the nearest power of two aligns CUTLASS and Triton implementations bit‑for‑bit and produces identical token sequences, with only minor perplexity changes.
By Teng-Ruei Chen
arXiv:2609.00555v1 Announce Type: cross
Abstract: Language-driven dexterous grasp models, such as DextER, perform well when instructions specify where to grasp, but we find they fail systematically w...
By Geonho Kim, SooGon Kim, Jongmin Lee
arXiv:2609.01200v1 Announce Type: new
Abstract: When the visual encoder and the language decoder of a vision-language model (VLM) run on different compute nodes, the intermediate visual-token embeddi...
By Reza Heidari, Hamed R. Tavakoli, Juho Kannala
arXiv:2411.15209v4 Announce Type: replace
Abstract: The expansion of time-series data from sensors and monitoring systems has made compact representations increasingly important. Such representations...
By Erin Carson, Xinye Chen, Fei He, Cheng Kang
The paper introduces SARTM, a framework that adapts the Segment Anything Model (SAM) for RGB‑thermal (RGB‑T) semantic segmentation. It fine‑tunes SAM with LoRA layers, incorporates language guidance, and employs a Cross‑Modal Knowledge Distillation module to bridge modality gaps. The approach also modifies the segmentation head and adds an auxiliary semantic head, achieving superior performance on MFNET, PST900, and FMB benchmarks.
By Dong Xing, Jinhe Zhang, Hang Yang, Yuqing Wang
The paper introduces CM-PTM, a Cross Multi-source Behavior Pre-Training Model designed to learn mobile game user representations from device-level behavioral logs. It uses hierarchical cascaded mask‑then‑predict tasks to first identify the source of the next behavior and then refine predictions at the app‑action level, thereby modeling cross‑source dependencies and fine‑grained dynamics. Experiments on large real‑world mobile datasets show that CM‑PTM captures users’ endogenous interests and improves performance on downstream mobile game recommendation tasks.
By Chengqi Yang, Yiran Qiao, Feng Liu, Xingyu Lou, Zijun Zhou, Xiaoyun Mo, Changwang Zhang, Jiayuan Xu, Jun Wang, Xiang Ao
arXiv:2602.09691v2 Announce Type: replace-cross
Abstract: Knowledge distillation (KD) is a technique to compress a larger teacher system into a smaller student. In machine translation, KD is commonly...
By Joseph Attieh, Timothee Mickus, Anne-Laure Ligozat, Aur\'elie N\'ev\'eol, J\"org Tiedemann
The paper investigates where post‑training quantization (PTQ) harms large language models (LLMs) and how to best allocate a limited precision budget. By causally raising each layer to 8‑bit precision across nine open‑weight models, the authors find that quantization damage is diffuse rather than concentrated in specific task circuits or weight statistics, and that globally refining quantization granularity outperforms selectively protecting the most recoverable layers. They also observe that the residual accuracy loss is budget‑limited and that peak recovery locations correlate with architecture within families but not across families.
By Jundong Hu, Shekar Ramachandran
MemoryWalker addresses the conditioning problem that arises when training agents with compressed context during rollout. It introduces two exact, gradient‑equivalent corrections—LogitTree, a segmented K‑forward traversal, and a packed 4D attention mask—alongside SDCC, a self‑distillation method that reduces the train‑deployment gap by minimizing KL divergence at each eviction. Experiments on seven web‑search benchmarks show that SDCC significantly lowers logit drift and boosts rollout rewards compared to naive training.
By Zinco J, Xunjie Zhu, Shen Huang, Zhenyi Wang, Pengjun Xie, Jieping Ye
arXiv:2609.00059v1 Announce Type: cross
Abstract: Materials property prediction remains difficult in low-data settings, where many target properties are supported by only a limited number of labeled...
By Weiran Wang, Xintong Huo, Yueying Wang, Yusi Fan, Wenyan Wang, Xin Feng, Ruihao Xin, Lan Huang, Kewei Li, Fengfeng Zhou
arXiv:2609.00066v1 Announce Type: cross
Abstract: NVFP4 is an efficient microscaling format for low-bit inference, but activation outliers can still degrade quantization accuracy within NVFP4 blocks....
By Yishan Yao, Binjun Li, Hanling Yi, Pengyu Li, Xiaoqing Liu, Zihan Yang, Xiaotian Yu, Zhiwen Yu
arXiv:2609.00901v1 Announce Type: new
Abstract: Modifying the illumination of driving images is a fundamental challenge, as most datasets are captured at specific times of day. Existing methods rely...
By Hala Djeghim, Nathan Piasco, Luis Rold\~ao, Moussab Bennehar, Dzmitry Tsishkou, C\'eline Loscos, D\'esir\'e Sidib\'e
arXiv:2603.10359v2 Announce Type: replace
Abstract: Distilling reasoning capabilities from Large Reasoning Models (LRMs) into smaller models is typically constrained by the limitations of rejection s...
By Wenjing Zhang, Jiangze Yan, Jieyun Huang, Yi Shen, Shuming Shi, Ping Chen, Ning Wang, Zhaoxiang Liu, Kai Wang, Shiguo Lian
arXiv:2609.00776v1 Announce Type: cross
Abstract: Digital interfaces are traditionally implemented through intermediate representations such as code, requiring their appearance and behavior to be spe...
By Yuval Alaluf, Omri Avrahami, Guy Bukchin Leshem, Michal Geyer, Kfir Goldberg, Elad Richardson, Diego Alarc\'on, Alejandro Alvarez, Cole Garry, Anastasis Germanidis, Tenaya Goldsen, Corina Gurau, Robin Kahlow, Joel Kwartler, Kathleen Lewis, Alejandro Matamala Ortiz, Eugene McMahon, Thon Prom, Sarah Saltonstall-Wurm, Jamie Umpherson, Hudson Yeo
ContextPipe is a database-inspired framework for assembling context in long-horizon large language model agents. It treats context assembly like relational query execution, using a five-phase pipeline—Plan, Bind, Optimize, Execute, Feedback—backed by a structured catalog, deterministic cache-aware optimizer, and EXPLAIN ANALYZE tracing. In a preliminary evaluation on the SWE-bench Pro Qutebrowser subset, ContextPipe reduced token volume by 31%, LLM calls by 23%, and response time by 9% compared to an append-only policy, though it lowered KV cache-hit ratio.
By Peng Xu, Zuyu Zhang, Yuze Sun, Feng Tian, Long Wang, Chen Zhang
The paper introduces Inverted Contrastive Learning for Unsupervised Feature Selection (ICLFS), a method that treats each feature as a sample by inverting the data matrix and applies a contrastive learning framework to learn consistent representations across masked positive views and a shuffled negative view. Feature saliency is derived from the magnitude of projector‑space embeddings, and a Laplacian‑Gated Ranking Correction step refines the ranking by reducing local redundancy. Experiments on 12 benchmark datasets show that ICLFS achieves the best clustering accuracy on 10 datasets compared to both classical and neural baselines, demonstrating the effectiveness of feature‑wise contrastive consistency for unsupervised feature selection.
By Utsab Ghosh, Roshni Chakraborty