arXiv Machine Learning

Joint End-to-End Training of Weighted BP Decoder Ensembles

The paper presents a method for jointly training an ensemble of weighted belief‑propagation decoders by optimizing trainable Tanner‑graph edge weights. Four ensemble objectives are compared, and best‑member training is found to be most effective. On a (64,32) PEG LDPC code, the ensemble achieves a 0.66 dB gain over a single decoder at FER = 10⁻⁴, and similar improvements are seen on a (128,64) CCSDS LDPC code, demonstrating learned complementary failure events rather than uniformly better components.

arXiv Machine Learning
Oct 23

On the Adversarial Vulnerability of Neural Receivers with Unrolled LDPC Decoders

The paper investigates the adversarial vulnerability of neural receivers that incorporate unrolled LDPC decoders for 5G NR. It finds that standard gradient-based attacks are ineffective due to gradient masking caused by tanh saturation in the BP check-node update, leading to a dramatic decay in gradient norm over iterations. To overcome this, the authors propose two loss functions—L_BP and L_LLR—that restore attack efficacy, significantly increasing block-error rates across different neural receiver implementations.

By Aakash Saini, Jafar Aco Mohammadi, Wolfgang Gerstacker
arXiv Machine Learning
Jul 14

Vertical Fusion: Condensing Internal Representations for Robust ViT Classification

arXiv:2607. 10391v1 Announce Type: cross Abstract: Despite exposing rich intermediate representations, Vision Transformers (ViTs) are almost exclusively utilized as black-box feature extractors, where only the last layer is considered for downstream tasks.

By Francesco Di Salvo, Shyam Nandan Rai, Hamed Damirchi, Ignacio Meza De la Jara, Sebastian Doerrich, Marco Lents, Christian Ledig
arXiv Machine Learning
Sep 21

On the Limits of Maximal Coding Rate Reduction for Out-of-Distribution Generalisation

The paper investigates the limits of the maximal coding rate reduction (MCR²) framework for out‑of‑distribution (OOD) generalisation. It shows that MCR² can lead to complete prediction failure under distribution shift, even when a perfectly stable feature is available, and that adding invariance principles from IRM or REx does not resolve this issue. The authors conclude that additional assumptions or learning principles are needed to guarantee stable OOD predictions with MCR².

By Menghui Zhou, Gaoshan Bi, Vitaveska Lanfranchi, Po Yang
arXiv AI
1d ago

Learning to Decode Linear Codes with Generalized Belief Propagation Network

The paper introduces GBPNet, a Generalized Belief Propagation Network that maintains message‑passing symmetry while improving learning capability for decoding linear codes. GBPNet employs cascaded variable‑node updates and learned element‑wise message scaling, preserving the Tanner graph structure. Two training strategies—progressive training and high‑SNR training—are proposed, and experiments show GBPNet achieves lower bit error rates than conventional belief propagation and other state‑of‑the‑art neural decoders.

By Samira Malek
arXiv Computer Vision
Sep 28

FuseReg: Regularizing Layer Fusion Mitigates the Reconstruction-Generation Gap in Representation Autoencoders

arXiv:2609.31620v1 Announce Type: new Abstract: Representation autoencoders (RAEs) reuse features from a pretrained visual encoder as reconstruction and diffusion latents, integrating strong visual r...

By Hongyang Du, Yunfei Xie, Junjie Ye, Jiawei Yang, Xiaoyan Cong, Haodong Zhang, Yongchao Huang, Haiyu Wu, Zongxia Li, Shihang Gui, Dawei Liu, Runhao Li, Jingcheng Ni, Chen Wei, Randall Balestriero, Yue Wang
arXiv AI
Sep 17

Beyond Truncation: Rethinking LLM Decoding as Ensemble Pruning

The paper introduces Mahalanobis-Ensemble Decoding (ME-Decoding), a new framework for Large Language Model decoding that treats candidate token selection as an ensemble pruning problem. It uses a Mahalanobis distance-driven objective to promote semantic diversity while maintaining high probabilities, employing a token similarity matrix built with an adaptive-bandwidth kernel over token embeddings. An efficient greedy algorithm with near-linear complexity and theoretical guarantees makes ME-Decoding a plug‑and‑play module with negligible inference overhead, and experiments show strong performance across reasoning and generation tasks.

By Dunyao Xue, Chengshuo Du, Zhengbo Wang, Wenlin Dai, Cheng Meng
arXiv Machine Learning
Jun 15

Ensembling Sparse Autoencoders

arXiv:2505. 16077v2 Announce Type: replace Abstract: Sparse autoencoders (SAEs) are used to decompose neural network activations into human-interpretable features.

By Soham Gadgil, Chris Lin, Su-In Lee