arXiv Machine Learning June 18, 2026 By Zhijie Deng, Ling Li, Jinlong Pang, Kaiqin Hu, Qi Xuan, Zhaowei Zhu, Jiaheng Wei

CODEBLOCK: Learning to Supervise Code at the Right Granularity

Read the original on arXiv Machine Learning →

arXiv:2606. 18286v1 Announce Type: new Abstract: Supervised fine-tuning of code LLMs typically applies uniform cross-entropy loss to all response tokens, implicitly assuming that every token provides equally useful learning signal.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Sep 4

Synthetic Semantic Supervision for Contrastive Code Representation Learning in Small Transformers: An Empirical Study

The paper investigates using synthetic natural-language descriptions to contrastively pretrain small transformer encoders for code representation. By pairing generated descriptions with code in a dual-encoder setup during training and discarding them at inference, the authors achieve significant improvements over traditional pretraining baselines on most evaluated tasks. When fine‑tuned, these models match or surpass much larger zero‑shot models and remain competitive with execution‑aware supervision, indicating a scalable alternative for code embeddings.

By Kenneth Paulsen, Florian Tambon, Mike Papadakis, Shin Yoo

llms rag fine-tuning benchmarks

arXiv Machine Learning

Jun 25

Weave of Formal Thought

arXiv:2606. 25987v1 Announce Type: cross Abstract: Large language models (LLMs) attain remarkable surface fluency on code, yet they neither formally guarantee the syntactic validity of their output nor leverage the hierarchical structure defining the target language.

By Alexandre Bouayad

llms nlp fine-tuning

arXiv AI

Jul 3

TokenScope: Token-Level Explainability and Interpretability for Code-Oriented Tasks in Large Language Models

arXiv:2607. 01235v1 Announce Type: cross Abstract: Understanding how Large Language Models (LLMs) make token-level decisions during code generation remains a major challenge for both researchers and practitioners.

By Amirreza Esmaeili, Fatemeh Fard

llms safety

arXiv AI

Jun 24

SemChunk-C: Semantic Segmentation for C Code

arXiv:2606. 23697v1 Announce Type: cross Abstract: Semantic segmentation of code written in a C-family language remains a challenging problem, due to the language's complex syntax, macro expansion, and irregular structural patterns.

By Boris Nazarov, Darya Frolova, Shaked Leibzirer, Pavel Kisilev

llms rag computer-vision benchmarks

arXiv Machine Learning

Sep 22

Token Signatures of Code: Comparing Coding Behaviors Across Large Language Models

arXiv:2609.22097v1 Announce Type: cross Abstract: The evaluation of large language models (LLMs) on coding tasks has primarily focused on performance metrics such as pass@k. As LLMs continue to advan...

By Junpeng Wang, Yuzhong Chen, Menghai Pan, Uday Singh Saini, Yiwei Cai

llms nlp

Hugging Face Trending Papers

Aug 5

OctoLong: Mid-Training On Cross-Repository Code Contexts Enhances Long-Context Modeling

Context lengths of language models (LMs) have dramatically increased, driven by the demands for in-context learning, self-improvement, and long-horizon agentic workflows. Existing long-context corpora, however, are dominated by books, academic articles, and code repositories, which are finite resources and often scarce in long-distance dependencies.

llms agents fine-tuning benchmarks

One email a morning, machine-written

Related stories

Synthetic Semantic Supervision for Contrastive Code Representation Learning in Small Transformers: An Empirical Study

Weave of Formal Thought

TokenScope: Token-Level Explainability and Interpretability for Code-Oriented Tasks in Large Language Models

SemChunk-C: Semantic Segmentation for C Code

Token Signatures of Code: Comparing Coding Behaviors Across Large Language Models

OctoLong: Mid-Training On Cross-Repository Code Contexts Enhances Long-Context Modeling