arXiv Machine Learning

Contextual Memory-Enhanced Source Coding for Low-SNR Communications

The paper introduces Memory-Augmented Source Coding (MASC), a scheme that embeds contextual patterns into a source model to improve robustness in low‑SNR communications. MASC uses a shared Parameterized Contextual Memory (PCM) for multi‑order n‑gram patterns and a Mixture‑of‑Memory‑Experts Router (MMER) to selectively activate memory experts based on hidden states, thereby refining probability estimates and shortening code length. Experiments on Rayleigh fading and AWGN channels show that MASC reduces decoding sensitivity to residual channel errors compared to traditional SSCC with autoregressive decoding and LLM‑based Arithmetic Coding.

arXiv Machine Learning
Aug 18

In-Context Source and Channel Coding

arXiv:2601. 10267v2 Announce Type: replace Abstract: Separate Source-Channel Coding (SSCC) remains attractive for text transmission due to its modularity and compatibility with mature entropy coders and powerful channel codes.

By Ziqiong Wang, Tianqi Ren, Rongpeng Li, Zhifeng Zhao, Honggang Zhang
arXiv Machine Learning
Jun 16

Context-Aware Markov VAE for CSI Compression in Wireless Systems

arXiv:2606. 16607v1 Announce Type: cross Abstract: This paper considers neural channel state information (CSI) compression for time-varying massive multiple-input multiple-output (MIMO) channels in frequency division duplex (FDD) systems with limited feedback resources.

By Efstathios Chatziloizos, Konstantinos Vandikas, Aneta Vulgarakis Feljan, Zheng Chen, Nikolaos Pappas
arXiv Computer Vision
Aug 31

Ada-TokenCom: Rate-Adaptive Token Communications via Large-Model-Driven Token Compression and Generation

Ada-TokenCom is a rate‑adaptive token communication framework that uses large autoregressive models to compress and transmit tokens efficiently. It combines next‑token prediction with arithmetic coding, sending only the most informative tokens and letting the receiver generate the rest. A Lyapunov‑based algorithm dynamically adjusts compression and modulation to match changing network conditions, and simulations show it outperforms existing digital and deep joint source‑channel coding baselines.

By Zijun Zhang, Li Qiao, Mahdi Boloursaz Mashhadi, Zhen Gao, Mehdi Bennis, Kaibin Huang
arXiv Machine Learning
4d ago

OMP-MoE: Efficient Expert Pruning for Mixture-of-Experts LLMs via Orthogonal Matching Pursuit

OMP-MoE is a training‑free compression framework that prunes redundant experts in Mixture‑of‑Experts large language models by framing the problem as sparse signal reconstruction solved with Orthogonal Matching Pursuit. The method greedily selects expert contributions as dictionary atoms to minimize reconstruction error, then optimizes cross‑layer expert allocation via a water‑filling strategy, and finally introduces an adaptive inference mechanism (OMP‑MoE†) that dynamically adjusts expert activation based on energy prediction. Experiments on Qwen, DeepSeek‑V2, GPT‑OSS, and Mixtral MoE show consistent performance gains at 25‑50% pruning ratios, with Qwen3‑30B‑A3B retaining 93.3% of original performance at 50% compression while achieving significant speedups.

By Dezhi Li, Lujun Li, Qiyuan Zhu, Hao Gu, Bei Liu, Sirui Han, Yike Guo