arXiv Machine Learning

LLM-Based Embeddings for Program Analysis and Optimization

arXiv:2608. 07894v1 Announce Type: new Abstract: Recent advances have highlighted the potential of machine learning, particularly Large Language Models (LLMs), for analyzing and optimizing programs.

arXiv AI

Aug 19

LSem2Vec: A Simple yet Effective Two-Stage Approach for Source Code Embedding

The paper introduces LSem2Vec, a two‑stage method that first uses a large language model to extract source code semantics and then applies a sentence embedding model to produce vector representations. This approach removes the need for task‑specific training or fine‑tuning, addressing errors in LLM outputs. Experiments on three datasets across multiple programming languages show that LSem2Vec outperforms five state‑of‑the‑art unsupervised methods.

By Zixiang Xian, Chenhui Cui, Rubing Huang, Chunrong Fang, Zhenyu Chen

llms rag fine-tuning benchmarks

arXiv AI

Sep 4

Synthetic Semantic Supervision for Contrastive Code Representation Learning in Small Transformers: An Empirical Study

The paper investigates using synthetic natural-language descriptions to contrastively pretrain small transformer encoders for code representation. By pairing generated descriptions with code in a dual-encoder setup during training and discarding them at inference, the authors achieve significant improvements over traditional pretraining baselines on most evaluated tasks. When fine‑tuned, these models match or surpass much larger zero‑shot models and remain competitive with execution‑aware supervision, indicating a scalable alternative for code embeddings.

By Kenneth Paulsen, Florian Tambon, Mike Papadakis, Shin Yoo

llms rag fine-tuning benchmarks

arXiv AI

Jun 16

FasterPy: An LLM-based Code Execution Efficiency Optimization Framework

arXiv:2512. 22827v2 Announce Type: replace-cross Abstract: Code often suffers from performance bugs.

By Yue Wu, Minghao Han, Ruiyin Li, Peng Liang, Amjed Tahir, Zengyang Li, Qiong Feng, Mojtaba Shahin

llms rag fine-tuning benchmarks

arXiv AI

Jul 13

Beyond Embeddings: Interpretable Feature Extraction for Binary Code Similarity

arXiv:2509. 23449v2 Announce Type: replace Abstract: Binary code similarity detection is a core task in reverse engineering.

By Charles E. Gagnon, Steven H. H. Ding, Philippe Charland, Benjamin C. M. Fung

llms rag agents benchmarks safety

arXiv AI

Jun 4

CodegenBench: Can LLMs Write Efficient Code Across Architectures?

arXiv:2606. 04023v1 Announce Type: cross Abstract: While large language models (LLMs) have been extensively evaluated on code generation tasks for general-purpose programming and GPU-accelerated environments (e.

By Jie Li, Wenzhao Wu, Junqi Hu, Qinrui Zheng, Bowen Wu, Juepeng Zheng, Yutong Lu, Haohuan Fu

llms benchmarks

arXiv AI

Jun 10

PromptEmbedder: Efficient and Transferable Text Embedding via Dual-LLM Soft Prompting

arXiv:2605. 28066v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have demonstrated remarkable efficacy in text embedding, yet current adaptation methods like LoRA face significant bottlenecks in computational efficiency and cross-architecture transferability.

By Yu-Che Tsai, Kuan-Yu Chen, Yuan-Hao Chen, Yu-Han Chang, Ching-Yu Tsai, Yu-Hsiang Chuang, Shou-De Lin

llms rag fine-tuning benchmarks safety

arXiv AI

Aug 18

Efficient Code Embeddings from Code Generation Models

arXiv:2508. 21290v2 Announce Type: replace-cross Abstract: jina-code-embeddings is a novel code embedding model suite designed to retrieve code from natural language queries, perform technical question-answering, and identify semantically similar code snippets across programming languages.

By Daria Kryvosheieva, Saba Sturua, Michael G\"unther, Han Xiao

rag benchmarks

arXiv Machine Learning

Jun 8

Skip a Layer or Loop It? Learning Program-of-Layers in LLMs

arXiv:2606. 06574v1 Announce Type: new Abstract: Large language models (LLMs) perform inference by following a fixed depth and order, non-recurrent execution of all layers.

By Ziyue Li, Yang Li, Tianyi Zhou

llms benchmarks

arXiv Machine Learning

Jun 25

LLM Program Optimization via Retrieval Augmented Search

arXiv:2501. 18916v2 Announce Type: replace Abstract: Recent work has demonstrated the potential of large language models (LLMs) for program optimization, a key challenge in programming languages.

By Sagnik Anupam, Alexander Shypula, Osbert Bastani

llms rag benchmarks safety

arXiv AI

Aug 11

Effect of Abstractions and Prompting Strategies on LLM-Guided High-Performance Optimizations

arXiv:2608. 08085v1 Announce Type: cross Abstract: Code performance optimization is a vital aspect of modern software development, as it enables faster response times and reduced resource usage.

By Ji\v{r}\'i Klepl, Maty'a\v{s} Brabec, Martin Kruli\v{s}

llms benchmarks

arXiv AI

Jun 24

SemChunk-C: Semantic Segmentation for C Code

arXiv:2606. 23697v1 Announce Type: cross Abstract: Semantic segmentation of code written in a C-family language remains a challenging problem, due to the language's complex syntax, macro expansion, and irregular structural patterns.

By Boris Nazarov, Darya Frolova, Shaked Leibzirer, Pavel Kisilev

llms rag computer-vision benchmarks

arXiv AI

Jul 31

ARC-Encoder: learning compressed text representations for large language models

arXiv:2510. 20535v2 Announce Type: replace-cross Abstract: Recent techniques such as retrieval-augmented generation or chain-of-thought reasoning have led to longer contexts and increased inference costs.

By Hippolyte Pilchen, Edouard Grave, Patrick P\'erez

llms rag fine-tuning benchmarks

Related stories