Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

31,046 stories · RSS feed

arXiv Machine Learning

Sep 30

Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning

arXiv:2609.37915v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) trains a student to match a privileged teacher distribution along its own sampled trajectory. Standard OPSD applies...

By Md. Ismail Hossain, Humaira Kousar, Isidora Chara Tourni

llms efficiency

arXiv Machine Learning

Sep 30

Pattern Formation in Transformers

arXiv:2609.37921v1 Announce Type: new Abstract: What are the inductive biases of a Transformer architecture? Existing theory on how the forward pass shapes representations either considers whether Tr...

By Erkan Turan, Gaspard Abel, Maks Ovsjanikov

llms

arXiv Machine Learning

Sep 30

TabFM-Auto: Self-Evolving Pipelines for Tabular Foundation Models

arXiv:2609.37989v1 Announce Type: new Abstract: Tabular foundation models achieve strong zero-shot accuracy on structured data by pretraining on synthetic tables, but they ignore the column names, ta...

By Deqing Fu, Huangyuan Su, Rajat Sen, Taman Narayan, Sujay Sanghavi, Abhimanyu Das, Weihao Kong

llms agents benchmarks data-engineering

arXiv Machine Learning

Sep 30

When do data mixtures improve scaling laws? Insights from high-dimensional regression

arXiv:2609.38011v1 Announce Type: new Abstract: Modern machine learning systems are trained on mixtures of data from different domains, and choosing the right mixture can substantially improve downst...

By Diyuan Wu, Lehan Chen, Theodor Misiakiewicz, Marco Mondelli

llms

arXiv AI

Sep 30

Dr. OPD: Learning What to Follow for Optimal On-Policy Distillation of Large Language Models

arXiv:2609.38025v1 Announce Type: cross Abstract: On-policy distillation (OPD) trains a student on its own generated responses using dense, token-level supervision from a stronger teacher. Vanilla OP...

By Zhenyu Wang, Tianze Wang, Linjun Zhang, Yifan Hu

llms efficiency

arXiv Machine Learning

Sep 30

Traversing the solution space of neural networks with Hessian Null Space Continuation

arXiv:2609.38081v1 Announce Type: new Abstract: On a single task, deep networks can learn many solutions, depending on their optimizer, training data, architecture, and hyperparameters. Many of these...

By Ann Huang, Mitchell Ostrow, Zhouyang Lu, William T. Redman, Leo Kozachkov, Kanaka Rajan

llms agents fine-tuning safety

arXiv Machine Learning

Sep 30

A Spectral Theory of Distortion in LLM Graph Reconstruction: Sharp Bounds and Empirical Characterization

arXiv:2609.38161v1 Announce Type: new Abstract: Evaluations of graph reconstruction by language models typically report a single aggregate distance between the original and the reconstructed graph. W...

By Jianru Shen

llms safety

arXiv Machine Learning

Sep 30

Sieve and Sage: Efficient Distraction Filtering for Reliable RALM Abstention

arXiv:2609.35794v1 Announce Type: cross Abstract: Just as Socrates recognized the limits of his own knowledge, Retrieval-Augmented Language Models (RALMs) should learn to abstain when the retrieved e...

By Jongbin Won, Sung Geun An, Jay-yoon Lee

llms rag safety

arXiv Machine Learning

Sep 30

Tracing mechanisms of sycophantic agreement in language models

arXiv:2609.35822v1 Announce Type: cross Abstract: Sycophantic agreement in language models refers to the tendency to overly affirm a user's stated beliefs or preferences, often at the expense of fact...

By Sixing Chen, Zhuofan Josh Ying, Logan Riggs Smith, Jeremy Wertheimer, Natalie Shapira

llms safety

arXiv AI

Sep 30

Same Bytes, Different Authority: Reserved-Token Representations in Chat-Template Prompt Injection

arXiv:2609.35932v1 Announce Type: cross Abstract: Prompt injection against LLM agents becomes much stronger when the injected instruction is wrapped in the model's own chat template. A forged templat...

By Yan Zhan, Yunze Song, Mengkai Hou, Wanting Zhang, Shaobo Liu, Zhijun Gao

llms rag agents nlp fine-tuning benchmarks

arXiv AI

Sep 30

Representation by Design in Generation: Cross-View Class-Token Alignment in Diffusion Transformers

arXiv:2609.36348v1 Announce Type: cross Abstract: Generative and representation learning remain asymmetrically connected: semantic representations are used to improve diffusion generation, whereas th...

By Xiaoyu Wu, Yifei Wang, Chen Wei

llms diffusion computer-vision safety

arXiv Machine Learning

Sep 30

Seeing What Should Be Heard: Diagnosing and Repairing Cross-Modal Shortcuts in Omni-Modal LLMs

arXiv:2609.36798v1 Announce Type: cross Abstract: Omni-modal large language models (LLMs) are expected to answer a question using the modality it explicitly refers to. However, existing training para...

By Yueran Ma, Ronghao Lin

llms nlp reinforcement-learning fine-tuning multimodal benchmarks

arXiv Machine Learning

Sep 30

You Cannot Recover What Was Never Measured: Quantifying the Information Ceiling of Ultra-Low-Field MRI Super-Resolution

arXiv:2609.36837v1 Announce Type: cross Abstract: Generative super-resolution models can turn portable 64 mT MRI into images that look like 3T scans, and the field evaluates them with PSNR, SSIM, and...

By Prathamesh Pradeep Khole, Shreya Handa, Utkarsh Gupta, Razvan Marinescu

llms diffusion benchmarks

arXiv AI

Sep 30

Safer Content or Firmer Refusals? A Hybrid Perturbation Defense for Alignment under Harmful Fine-tuning

arXiv:2609.36862v1 Announce Type: cross Abstract: Fine-tuning-as-a-service lets users adapt a safety-aligned language model to their own data, but it also creates a harmful fine-tuning attack surface...

By Muhammad Zeeshan Akram, Mufid Kamel Marican, Anvesh Reddy Yenugu, Ali Zain Kaimkhani, Minghong Fang

llms rag fine-tuning safety

arXiv Machine Learning

Sep 30

GleanVID: Complementary Token Selection for Efficient Video Large Language Models

arXiv:2609.37042v1 Announce Type: cross Abstract: Video Large Language Models (VideoLLMs) have achieved strong video understanding capabilities but incur substantial inference overhead due to the lar...

By Shuo Yang, Changbai Li, Rui Tang, Xinyu Zhao, Linlin Yang, Baochang Zhang

llms benchmarks

arXiv Machine Learning

Sep 30

vSkipper: Translating Dynamic Layer Skipping into LLM Serving Gains

arXiv:2609.37062v1 Announce Type: cross Abstract: Dynamic layer skipping reduces LLM computation by allowing each token to execute only a subset of the model's layers. However, existing skippers rely...

By Wei Da, Yavuz Ferhatosmanoglu, Evangelia Kalyvianaki

llms efficiency

arXiv Machine Learning

Sep 30

Improved Distributional Diffusion Models

arXiv:2609.37147v1 Announce Type: cross Abstract: Distributional Diffusion Models (DDMs) replace the standard mean-prediction denoiser with a \emph{distributional} denoiser trained via a scoring rule...

By Tommaso Martorella, Alexandre Galashov, Felix Krause, Stefan Andreas Baumann, Valentin De Bortoli, Arthur Gretton, Bj\"orn Ommer

llms diffusion efficiency

arXiv AI

Sep 30

Follow the Entities: A Corpus Map for Agentic Search

arXiv:2609.37226v1 Announce Type: cross Abstract: Answering questions and completing tasks over large document collections often requires connecting evidence spread across multiple documents, such as...

By Soyeong Jeong, Sujay Kumar Jauhar, Sung Ju Hwang, Andrew Joohun Nam

llms agents benchmarks

arXiv Machine Learning

Sep 30

SafeLLM4SE: Statistical Evaluation and Reporting for LLM-based Software Engineering Systems

arXiv:2609.37294v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for software engineering tasks, yet their stochastic behavior challenges the validity, reproducibi...

By Francisco Ortin

llms benchmarks

arXiv AI

Sep 30

MetaCtrl: Your Large Language Models Can Reason Better and More Concisely with a Metacognitive Controller

arXiv:2609.37304v1 Announce Type: new Abstract: Large reasoning models improve performance on challenging problems by allocating additional computation before answering, but longer reasoning does not...

By Zhibin Wen, Tao Han, Lei Bai, Can Li, Yang Xu

llms reinforcement-learning benchmarks

Get the daily briefing