arXiv Machine Learning By Rapha\"el Le Bidan, Ahmad Ismail, Elsa Dupraz, Charbel Abdel Nour

Leveraging Code Automorphisms for Improved Syndrome-Based Neural Decoding

Read the original on arXiv Machine Learning →

The paper demonstrates that incorporating code automorphisms into syndrome-based neural decoding (SBND) improves the models’ learning and generalization through data augmentation during training and inference. By applying this technique to short, high-rate codes, the authors achieve performance close to maximum likelihood decoding (MLD) using small datasets and appropriate training. The study also indicates that previous SBND results may have underestimated their true error‑correction capability due to insufficient training.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI
Aug 18

Invariant Pretraining for Robust Code Representations

arXiv:2608. 15412v1 Announce Type: cross Abstract: Encoder-based code representation models remain widely deployed for discriminative tasks such as clone detection and code classification, where their small size and low inference cost are decisive.

By Yifeng He, Yundi Xu, Christopher Castro Gaw Gonzalo, Zili Wang, Hao Chen
arXiv AI
Sep 4

Synthetic Semantic Supervision for Contrastive Code Representation Learning in Small Transformers: An Empirical Study

The paper investigates using synthetic natural-language descriptions to contrastively pretrain small transformer encoders for code representation. By pairing generated descriptions with code in a dual-encoder setup during training and discarding them at inference, the authors achieve significant improvements over traditional pretraining baselines on most evaluated tasks. When fine‑tuned, these models match or surpass much larger zero‑shot models and remain competitive with execution‑aware supervision, indicating a scalable alternative for code embeddings.

By Kenneth Paulsen, Florian Tambon, Mike Papadakis, Shin Yoo