arXiv Machine Learning

Token Signatures of Code: Comparing Coding Behaviors Across Large Language Models

arXiv AI

Jul 3

TokenScope: Token-Level Explainability and Interpretability for Code-Oriented Tasks in Large Language Models

arXiv:2607. 01235v1 Announce Type: cross Abstract: Understanding how Large Language Models (LLMs) make token-level decisions during code generation remains a major challenge for both researchers and practitioners.

By Amirreza Esmaeili, Fatemeh Fard

llms safety

arXiv AI

Aug 17

The Fools are Certain; the Wise are Doubtful: Exploring LLM Confidence in Code Completion

arXiv:2508. 16131v3 Announce Type: replace-cross Abstract: Code completion entails the task of providing missing tokens given a surrounding context.

By Zoe Kotti, Konstantina Dritsa, Diomidis Spinellis, Panos Louridas

llms fine-tuning safety

arXiv AI

Jun 9

Lost in the Flow with Code Talkers: Unveiling the Instruction-Tuning Tax of Large Language Models in Code Tasks

arXiv:2606. 08676v1 Announce Type: cross Abstract: AI coding assistants have significantly improved developer productivity by automatically suggesting code that aligns with user intent, and many of these tools are now integrated directly into Integrated Development Environments (IDEs).

By Shi Ying Chang, Chiok Yew Ho, Yichen Li, Yintong Huo

llms fine-tuning

arXiv AI

Aug 11

Contamination Means Overestimation? A Fine-Grained Empirical Study in Code Intelligence

arXiv:2506. 02791v4 Announce Type: replace-cross Abstract: In recent years, code intelligence has gained increasing importance in the field of automated software engineering.

By Zhen Yang, Hongyi Lin, Yifan He, Junqi Wang, Zeyu Sun, Shuo Liu, Jie Xu, Pengpeng Wang, Zhongxing Yu, Qingyuan Liang

llms nlp fine-tuning

arXiv Machine Learning

Jul 14

Sense and Sensitivity: Examining the Influence of Semantic Recall on Long Context Code Understanding

arXiv:2505. 13353v5 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly deployed for understanding large codebases, but whether they understand operational semantics of long code context or rely on pattern matching shortcuts remains unclear.

By Adam \v{S}torek, Mukur Gupta, Samira Hajizadeh, Prashast Srivastava, Suman Jana

llms benchmarks

arXiv AI

Aug 18

Invariant Pretraining for Robust Code Representations

arXiv:2608. 15412v1 Announce Type: cross Abstract: Encoder-based code representation models remain widely deployed for discriminative tasks such as clone detection and code classification, where their small size and low inference cost are decisive.

By Yifeng He, Yundi Xu, Christopher Castro Gaw Gonzalo, Zili Wang, Hao Chen

arXiv AI

2d ago

Complexity-Aware Evaluation of LLM Comprehension

arXiv:2609.37405v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for software engineering tasks that require understanding existing source code, including behavior...

By Ali Mohammadi Esfahani, Nafiseh Kahani, Samuel A. Ajila

llms benchmarks

arXiv Machine Learning

Aug 11

Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring

arXiv:2605. 00754v4 Announce Type: replace-cross Abstract: Reward models (RMs) have become an indispensable fixture of the language model (LM) post-training playbook, enabling policy alignment and test-time scaling.

By Indraneil Paul, Goran Glava\v{s}, Iryna Gurevych

llms reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 16

Exploring Extrinsic and Intrinsic Properties for Effective Reasoning with Code Interpreter

arXiv:2606. 16934v1 Announce Type: cross Abstract: Reasoning with a Code Interpreter (CI) has emerged as an effective paradigm for enhancing the reasoning capabilities of large language models (LLMs) through executable computation and iterative verification.

By Patomporn Payoungkhamdee, Napat Laosaengpha, Jenta Wonglertsakul, Pittawat Taveekitworachai, Pume Tuchinda, Panjapong Poobanchuen, Ekapol Chuangsuwanich, Can Udomcharoenchaikit, Samuel Cahyawijaya, Peerat Limkonchotiwat, Sarana Nutanong

llms reinforcement-learning fine-tuning benchmarks

arXiv Computer Vision

2d ago

Evaluating Generative Models via One-Dimensional Code Distributions

arXiv:2603.08064v3 Announce Type: replace Abstract: Most evaluations of generative models rely on feature-distribution metrics such as FID, which operate on continuous recognition features that are e...

By Zexi Jia, Pengcheng Luo, Yijia Zhong, Jinchao Zhang, Jie Zhou

benchmarks

arXiv Machine Learning

Jun 18

CODEBLOCK: Learning to Supervise Code at the Right Granularity

arXiv:2606. 18286v1 Announce Type: new Abstract: Supervised fine-tuning of code LLMs typically applies uniform cross-entropy loss to all response tokens, implicitly assuming that every token provides equally useful learning signal.

By Zhijie Deng, Ling Li, Jinlong Pang, Kaiqin Hu, Qi Xuan, Zhaowei Zhu, Jiaheng Wei

llms fine-tuning benchmarks

arXiv AI

Sep 4

Synthetic Semantic Supervision for Contrastive Code Representation Learning in Small Transformers: An Empirical Study

The paper investigates using synthetic natural-language descriptions to contrastively pretrain small transformer encoders for code representation. By pairing generated descriptions with code in a dual-encoder setup during training and discarding them at inference, the authors achieve significant improvements over traditional pretraining baselines on most evaluated tasks. When fine‑tuned, these models match or surpass much larger zero‑shot models and remain competitive with execution‑aware supervision, indicating a scalable alternative for code embeddings.

By Kenneth Paulsen, Florian Tambon, Mike Papadakis, Shin Yoo

llms rag fine-tuning benchmarks

Related stories