CDAE: Enhancing Perturbation Robustness in Pretrained Language Models with Contrastive Denoising

The paper introduces CDAE, a lightweight Contrastive Denoising Autoencoder that refines pre-trained BERT embeddings by jointly optimizing contrastive and reconstruction objectives. It aims to produce perturbation‑invariant sentence representations, addressing sensitivity to textual perturbations such as synonym substitution, masking, and word dropout. Experiments across various perturbation strategies and strengths show that CDAE consistently maintains higher embedding similarity than original BERT and SimCSE, with the advantage growing as perturbation strength increases.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• rag