arXiv AI
Jul 3
XSearch: Explainable Code Search via Concept-to-Code Alignment
arXiv:2605. 16046v2 Announce Type: replace-cross Abstract: Semantic code search has been widely adopted in both academia and industry.
MAP4CS: A Multi-dimensional Data Pruning Framework for Efficient Code Retriever Fine-tuning
MAP4CS is a data pruning framework that selects a small, high‑quality core subset from large code corpora by combining syntactic structure, semantic diversity, and distributional representation, followed by rule‑based filtering. Experiments on two large datasets show that using only 5% of the training data, MAP4CS outperforms random sampling and matches or exceeds full‑corpus fine‑tuning, supporting the idea that less data can be more effective. The method also automatically removes duplicates and noise, producing a lexically diverse and information‑dense training set.
Related stories
arXiv AI
Oct 2
Are AI Coders Snitches? An Empirical Study of Pretraining Data Detection on Code Large Language Models
The paper investigates whether code large language models (CodeLLMs) inadvertently reproduce proprietary or sensitive code by evaluating seven state‑of‑the‑art training data detection (TDD) methods on eight CodeLLMs. It introduces CodeSnitch, a benchmark of 9,000 function‑level code samples across three languages, each labeled as included or excluded from training data, and applies mutation strategies based on the Type‑1 to Type‑4 code clone taxonomy to test TDD robustness. The study offers a systematic assessment of current TDD techniques for code and suggests directions for developing more effective detection methods.
arXiv Machine Learning
Jul 30
Train Large, Deploy Compact: Structured Compression for Compact Low-Rank Adaptation
arXiv:2510. 00192v3 Announce Type: replace Abstract: Low-rank adaptation (LoRA) has become a widely used paradigm for parameter-efficient fine-tuning of large language models, yet its representational capacity often lags behind full fine-tuning.
arXiv AI
Jul 28
CuraWeb: Joint Optimization of Quality, Redundancy, and Diversity for Web-Scale Pretraining Data
arXiv:2607. 22662v1 Announce Type: new Abstract: Open-web corpora curated via highly selective filters, such as FineWeb-Edu and DCLM, constitute the core of LLM pretraining data and have significantly advanced LLM performance.
arXiv Computation and Language
Aug 21
Natural Language Code Retrieval for 1C:Enterprise: An Open Benchmark and Efficient Bi-Encoder
arXiv:2608. 19957v1 Announce Type: new Abstract: Natural language code retrieval is a rapidly evolving task in computer science.
arXiv AI
Jun 8
CoQuIR: A Comprehensive Benchmark for Code Quality-Aware Information Retrieval
arXiv:2506. 11066v3 Announce Type: replace-cross Abstract: Code retrieval is essential in modern software development, as it boosts code reuse and accelerates debugging.
arXiv AI
Jul 22
Large Language Models Explore by Latent Distilling
arXiv:2604. 24927v2 Announce Type: replace-cross Abstract: Generating diverse responses is crucial for test-time scaling of large language models (LLMs), yet standard stochastic sampling mostly yields surface-level lexical variation, limiting semantic exploration.
arXiv Machine Learning
Aug 4
LangFIR: Discovering Sparse Language-Specific Features from Monolingual Data for Language Steering
arXiv:2604. 03532v2 Announce Type: replace-cross Abstract: Large language models (LLMs) show strong multilingual capabilities, yet reliably controlling the language of their outputs remains difficult.
arXiv AI
Oct 5
Are you Synthesizing or Recalling? Evaluating LLMs on Algorithmic Code Retrieval
arXiv:2610.02438v1 Announce Type: cross Abstract: Large language models (LLMs) have demonstrated strong performance in code generation, where success depends on both recalling relevant algorithmic kn...
arXiv AI
Jun 16
AlignCoder: Aligning Retrieval with Target Intent for Repository-Level Code Completion
arXiv:2601. 19697v2 Announce Type: replace-cross Abstract: Repository-level code completion remains a challenging task for existing code large language models (code LLMs) due to their limited understanding of repository-specific context and domain knowledge.
arXiv AI
Sep 15
CHAI for LLMs: Improving Code-Mixed Translation in Large Language Models through Reinforcement Learning with AI Feedback
CHAI for LLMs is a framework that improves large language models’ performance on code‑mixed translation tasks by using LLMs as annotators to create preference data, applying reinforcement learning from AI feedback, incorporating LLM‑generated domain knowledge for iterative refinement, and evaluating on real‑world datasets. The approach yields a 68.45% average win rate over state‑of‑the‑art open‑source models in human‑adjudicated tests. It demonstrates a scalable method to enhance code‑mixed language understanding in open‑source LLMs.
arXiv Computation and Language
Aug 31
Pruning Laws for Large Language Models
arXiv:2504.04342v2 Announce Type: replace Abstract: Scaling up model parameters and training data consistently improves the performance of large language models (LLMs), but at the cost of rapidly gro...