Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

31,384 stories · RSS feed

arXiv Machine Learning

Jun 2

ForesightKV: Optimizing KV Cache Eviction for Reasoning Models by Learning Long-Term Contribution

arXiv:2602. 03203v2 Announce Type: replace-cross Abstract: Recently, large language models (LLMs) have shown remarkable reasoning abilities by producing long reasoning traces.

By Zican Dong, Peiyu Liu, Junyi Li, Zhipeng Chen, Han Peng, Shuo Wang, Wayne Xin Zhao

llms reinforcement-learning efficiency benchmarks

arXiv AI

Jun 2

OGLS-SD: On-Policy Self-Distillation with Outcome-Guided Logit Steering for LLM Reasoning

arXiv:2605. 12400v2 Announce Type: replace-cross Abstract: We study on-policy self-distillation (OPSD), where a language model improves its reasoning ability by distilling privileged teacher distributions along its own on-policy trajectories.

By Yuxiao Yang, Xiaoyun Wang, Weitong Zhang

llms efficiency benchmarks

arXiv Machine Learning

Jun 2

LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding

arXiv:2602. 23881v2 Announce Type: replace Abstract: Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to propose candidate tokens that are then verified in parallel by the target model.

By Alexander Samarin, Sergei Krutikov, Anton Shevtsov, Sergei Skvortsov, Filipp Fisin, Alexander Golubev

llms

arXiv Machine Learning

Jun 2

Vegas: Self-Speculative Decoding with Verification-Guided Sparse Attention

arXiv:2602. 07223v2 Announce Type: replace Abstract: Long-context large language model (LLM) inference has become the norm for today's AI applications.

By Yikang Yue, Yuqi Xue, Jian Huang

llms efficiency benchmarks

arXiv AI

Jun 2

Geometric Erasure by Contrastive Velocity Matching in Rectified Flows

arXiv:2606. 00140v1 Announce Type: cross Abstract: While the rapid adoption of multimodal generative models offers immense potential, it has also increased the risks of harmful content synthesis, deepfakes, and copyright infringements.

By Jonas Henry Grebe, Tobias Braun, Anna Rohrbach, Marcus Rohrbach

llms diffusion multimodal

arXiv Machine Learning

Jun 2

Task Structure Reverses Layerwise State Encoding in Sequence Models

arXiv:2606. 00926v1 Announce Type: new Abstract: Mechanistic studies of sequence models often treat layerwise state encodings as architectural traits: recurrent models concentrate readable state, attention-based models distribute it.

By Yuhang Jiang

llms fine-tuning benchmarks

arXiv Machine Learning

Jun 2

CRMA: A Spectrally-Bounded Backbone for Modular Continual Fine-Tuning of LLMs

arXiv:2606. 00382v1 Announce Type: new Abstract: Sequential fine-tuning of large language models forces a choice: let the shared substrate keep learning and accept catastrophic forgetting, or freeze it after task one and foreclose cross-task refinement.

By Kiran Nayudu, Aswini Nutakki, Sai Vinay Naidu, Ashwin Shanmugasundaram

llms fine-tuning efficiency benchmarks

arXiv AI

Jun 2

Normalization Equivariance for Arbitrary Backbones, with Application to Image Denoising

arXiv:2605. 08193v3 Announce Type: replace-cross Abstract: Normalization Equivariance (NE) is a structural prior that improves robustness to distribution shift in image-to-image tasks.

By Youssef Saied, Fran\c{c}ois Fleuret

llms

arXiv AI

Jun 2

Understand and Accelerate Memory Processing Pipeline for Large Language Model Inference

arXiv:2603. 29002v3 Announce Type: replace-cross Abstract: Modern large language models (LLMs) increasingly depends on efficient long-context processing and generation mechanisms, including sparse attention, retrieval-augmented generation (RAG), and compressed contextual memory, to support complex reasoning.

By Zifan He, Rui Ma, Yizhou Sun, Jason Cong

llms rag

arXiv AI

Jun 2

Limits of Spatial Imagery Reasoning in Frontier LLM Models

arXiv:2603. 26779v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, yet they struggle with spatial tasks that require mental simulation, such as mental rotation.

By Sergio Y. Hayashi, Nina S. T. Hirata

llms

arXiv Machine Learning

Jun 2

Massive Spikes in LLMs are Bias Vectors: Mechanistic Uncovering and Spike-Free Quantization

arXiv:2606. 02288v1 Announce Type: new Abstract: Massive activation spikes in Large Language Models (LLMs) severely degrade quantization by stretching dynamic ranges.

By Yung-Chin Chen, Chung Peng Lee, Ze-Wei Liou, Naveen Verma

llms rag efficiency benchmarks safety

arXiv AI

Jun 2

TukaBench: A Culturally Grounded Jailbreak Benchmark for African Languages

arXiv:2606. 01322v1 Announce Type: cross Abstract: Safety evaluation of Large Language Models (LLMs) remains heavily English-centric, leaving Low-Resource Languages (LRLs), particularly African ones, critically underexplored.

By Victor Akinode, Senyu Li, Wassim Hamidouche, Waqas Zamir, Inbal Becker-Reshef, David Ifeoluwa Adelani

llms benchmarks safety

arXiv AI

Jun 2

Latent Reward Steering: An Adaptive Inference-Time Framework that Implicitly Promotes Cognitive Behaviors in Reasoning LLMs

arXiv:2606. 00726v1 Announce Type: new Abstract: Strong reasoning depends not only on model knowledge but also on how effectively cognitive behaviors are deployed during generation.

By Jiakang Li, Guanyu Zhu, Can Jin, Chenxi Huang, Dexu Yu, Ronghao Chen, Yang Zhou, Hongwu Peng, Xuanqi Lan, Dimitris N. Metaxas, Youhua Li

llms reinforcement-learning benchmarks

arXiv AI

Jun 2

Probe Before You Edit: Probing-Guided Molecular Optimization for LLM Agents in Structure-Based Drug Design

arXiv:2606. 00555v1 Announce Type: new Abstract: Structure-based drug design increasingly employs LLM agents to iteratively refine ligands against a target pocket, yet a viable ligand must satisfy two often-conflicting objectives -- binding affinity and druggability -- which single optimization steps rarely improve together.

By Zaifei Yang, Weiyu Chen, Yaqing Wang, James Kwok

llms agents benchmarks

arXiv AI

Jun 2

Revisiting Ripple Effects in Knowledge Editing through Pressure-Aware Joint Neighborhood Optimization

arXiv:2606. 01610v1 Announce Type: new Abstract: Single-edit updates in large language models can trigger ripple effects across local knowledge neighborhoods: desirable propagation to related facts and unintended perturbation of preserved ones.

By Haoben Huang, Shuxin Liu, Ou Wu, Di Gao

llms

arXiv Machine Learning

Jun 2

Feature to Dynamics: Feature-space to Autoregression strategy for Zero-shot Time Series Forecasting

arXiv:2606. 01289v1 Announce Type: new Abstract: Zero-shot time series forecasting aims to predict future values for previously unseen series, requiring models to generalize temporal dynamics beyond the training distribution.

By Yifan Wu, Junjie Wu, Kai Wu, Xiaoyu Zhang, Jian Lou

llms

arXiv AI

Jun 2

Principle-Evolvable Scientific Discovery via Uncertainty Minimization

arXiv:2602. 06448v2 Announce Type: replace-cross Abstract: Large Language Model (LLM)-based scientific agents have accelerated scientific discovery, yet they often suffer from significant inefficiencies due to adherence to fixed initial priors.

By Yingming Pu, Tao Lin, Hongyu Chen

llms agents benchmarks

arXiv AI

Jun 2

Algorithmic Fragility and Persona Bias in LLM-Generated Autistic Communication

arXiv:2605. 26397v2 Announce Type: replace-cross Abstract: Safety alignment reduces explicitly harmful outputs but inadvertently encodes a sanitized, neuronormative representation of marginalized communication.

By Naba Rizvi, Mohammed Rizvi, Harper Strickland, Saleha Ahmedi, Nedjma Ousidhoum

llms agents safety

arXiv Machine Learning

Jun 2

Spatiotemporal Multi-Task Graph Transformer for Trip-Level Transit Prediction

arXiv:2606. 00572v1 Announce Type: new Abstract: Passenger count data from public transit systems reveals urban mobility patterns and is essential for planning, operation, and optimisation.

By Oluwaleke Yusuf, Adil Rasheed, Frank Lindseth

llms rag benchmarks safety

arXiv Machine Learning

Jun 2

Semantic-Geometric Task Representations for Bimanual Manipulation from Human Demonstrations to Robot Action Planning

arXiv:2601. 11460v2 Announce Type: replace-cross Abstract: Learning structured task representations from human demonstrations is essential for bimanual manipulation, where action ordering, object involvement, and interaction geometry vary significantly across executions.

By Franziska Herbert, Vignesh Prasad, Han Liu, Dorothea Koert, Georgia Chalvatzaki

llms robotics fine-tuning multimodal

Get the daily briefing