Text and code embeddings by contrastive pre-training

Efficient Code Embeddings from Code Generation Models

arXiv:2508. 21290v2 Announce Type: replace-cross Abstract: jina-code-embeddings is a novel code embedding model suite designed to retrieve code from natural language queries, perform technical question-answering, and identify semantically similar code snippets across programming languages.

Invariant Pretraining for Robust Code Representations

arXiv:2608. 15412v1 Announce Type: cross Abstract: Encoder-based code representation models remain widely deployed for discriminative tasks such as clone detection and code classification, where their small size and low inference cost are decisive.

LSem2Vec: A Simple yet Effective Two-Stage Approach for Source Code Embedding

arXiv:2409. 14644v4 Announce Type: replace-cross Abstract: The advent of large language models (LLMs) has significantly advanced artificial intelligence in software engineering, with source code embeddings playing a crucial role in tasks such as source code clone detection and source code clustering.

LLM-Based Embeddings for Program Analysis and Optimization

arXiv:2608. 07894v1 Announce Type: new Abstract: Recent advances have highlighted the potential of machine learning, particularly Large Language Models (LLMs), for analyzing and optimizing programs.

BitNet Text Embeddings

LLM-based text embedders have substantially improved retrieval and semantic representation quality, but their deployment remains costly: large backbone models slow down embedding inference, while high-dimensional full-precision embeddings impose substantial storage and bandwidth overhead on large-scale indexes. In this paper, we present BITEMBED, an extreme low-bit framework for LLM-based text embedding that jointly targets encoding efficiency and vector storage.

Contrastive vision-language learning with paraphrasing and negation

arXiv:2511. 16527v2 Announce Type: replace-cross Abstract: Contrastive vision-language models continue to be the dominant approach for image-text retrieval.

Structuring Semantic Embeddings for Principle Evaluation: A Prototype-Guided Contrastive Learning Approach

arXiv:2608. 15224v1 Announce Type: new Abstract: Reliable post-hoc evaluation asks whether already generated text satisfies a target criterion after generation.

OctoLong: Mid-Training On Cross-Repository Code Contexts Enhances Long-Context Modeling

Context lengths of language models (LMs) have dramatically increased, driven by the demands for in-context learning, self-improvement, and long-horizon agentic workflows. Existing long-context corpora, however, are dominated by books, academic articles, and code repositories, which are finite resources and often scarce in long-distance dependencies.

Evaluating large language models trained on code

DenseOn with the LateOn: Fully Open Dense and Late-Interaction Models for Multilingual, Long-Context, and Code Search

State-of-the-art retrieval models increasingly rely on closed training data, creating a reproducibility gap. We present an open end-to-end recipe for training retrieval models and study how English supervision transfers to multilingual retrieval through translate-train.

Scalable Visual Pretraining for Language Intelligence

arXiv:2607. 09657v1 Announce Type: cross Abstract: The rapid progress of large foundation models has been driven predominantly by pretraining on large-scale text corpora.

STEB: Style Text Embedding Benchmark

arXiv:2606. 31741v1 Announce Type: cross Abstract: While semantic embeddings are rigorously evaluated on the Massive Text Embedding Benchmark, the evaluation of style embeddings remains fragmented, with each work relying on their own set of tasks and datasets.