arXiv Machine Learning

Type-IV Code Clone Detection via Layer-Wise Non-Contrastive Representation Learning

The paper introduces LWVIC4Code, a non‑contrastive, layer‑wise representation learning method for detecting Type‑IV code clones—semantically equivalent fragments that differ syntactically. It builds on the VICReg framework, adding cross‑layer consistency regularization and depth‑dependent weighting to refine semantic information across transformer layers. Experiments on Python and multi‑language datasets show that LWVIC4Code matches or outperforms contrastive baselines and zero‑shot large language models, generalizing well to Java and C# without requiring negative samples.

arXiv AI
Aug 18

Invariant Pretraining for Robust Code Representations

arXiv:2608. 15412v1 Announce Type: cross Abstract: Encoder-based code representation models remain widely deployed for discriminative tasks such as clone detection and code classification, where their small size and low inference cost are decisive.

By Yifeng He, Yundi Xu, Christopher Castro Gaw Gonzalo, Zili Wang, Hao Chen
arXiv AI
Oct 2

Are AI Coders Snitches? An Empirical Study of Pretraining Data Detection on Code Large Language Models

The paper investigates whether code large language models (CodeLLMs) inadvertently reproduce proprietary or sensitive code by evaluating seven state‑of‑the‑art training data detection (TDD) methods on eight CodeLLMs. It introduces CodeSnitch, a benchmark of 9,000 function‑level code samples across three languages, each labeled as included or excluded from training data, and applies mutation strategies based on the Type‑1 to Type‑4 code clone taxonomy to test TDD robustness. The study offers a systematic assessment of current TDD techniques for code and suggests directions for developing more effective detection methods.

By Tianlin Li, Yunxiang Wei, Zhiming Li, Aishan Liu, Qing Guo, Xianglong Liu, Dongning Sun, Yang Liu
arXiv AI
Sep 4

Synthetic Semantic Supervision for Contrastive Code Representation Learning in Small Transformers: An Empirical Study

The paper investigates using synthetic natural-language descriptions to contrastively pretrain small transformer encoders for code representation. By pairing generated descriptions with code in a dual-encoder setup during training and discarding them at inference, the authors achieve significant improvements over traditional pretraining baselines on most evaluated tasks. When fine‑tuned, these models match or surpass much larger zero‑shot models and remain competitive with execution‑aware supervision, indicating a scalable alternative for code embeddings.

By Kenneth Paulsen, Florian Tambon, Mike Papadakis, Shin Yoo
arXiv AI
Aug 19

LSem2Vec: A Simple yet Effective Two-Stage Approach for Source Code Embedding

The paper introduces LSem2Vec, a two‑stage method that first uses a large language model to extract source code semantics and then applies a sentence embedding model to produce vector representations. This approach removes the need for task‑specific training or fine‑tuning, addressing errors in LLM outputs. Experiments on three datasets across multiple programming languages show that LSem2Vec outperforms five state‑of‑the‑art unsupervised methods.

By Zixiang Xian, Chenhui Cui, Rubing Huang, Chunrong Fang, Zhenyu Chen
arXiv AI
Oct 19

Pluvio: Assembly Clone Search for Out-of-domain Architectures and Libraries through Transfer Learning and Conditional Variational Information Bottleneck

arXiv:2307.10631v2 Announce Type: replace-cross Abstract: The practice of code reuse is crucial in software development for a faster and more efficient development lifecycle. In reality, however, cod...

By Zhiwei Fu, Steven H. H. Ding, Furkan Alaca, Benjamin C. M. Fung, Philippe Charland
arXiv Machine Learning
Oct 21

LLMs are Brittle to Simple Code Transformations: Introducing CETBench - A Benchmark for Code-Equivalence Checking

The paper introduces CETBench, a benchmark for assessing large language models (LLMs) on code-equivalence checking by generating program pairs through random code transformations. It demonstrates that even simple transformations significantly degrade LLM performance, especially in cross-lingual scenarios, and proposes a fine-tuning approach to improve results. The study provides detailed ablations on program difficulty and transformation types, offering insights into LLMs’ limited semantic understanding of code.

By Neeva Oza, Ishaan Govil, Parul Gupta, Dinesh Khandelwal, Dinesh Garg, Parag Singla
arXiv Computation and Language
Oct 16

Enhancing Knowledge Density via Hierarchical Context Pruning for Retrieval-Augmented Repository-Level Code Completion

The paper introduces SaraCoder, a retrieval‑augmented framework that improves knowledge density for repository‑level code completion by pruning redundant context and selectively injecting cross‑file dependency definitions within a fixed token budget. It employs a Hierarchical Feature Optimization module to reduce an over‑complete candidate pool in four stages, freeing tokens for an External‑Aware Identifier Disambiguator. Experiments on CrossCodeEval and RepoEval‑Updated demonstrate that SaraCoder outperforms strong RAG baselines such as GraphCoder and Repocoder across multiple code LLMs without increasing token usage, and achieves synergistic gains when combined with other cross‑file methods.

By Xiaohan Chen, Xiang Chen, Yu Tian, Piji Li, Sheng-Jun Huang, Songcan Chen