Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

30,670 stories · RSS feed

arXiv Machine Learning

Sep 30

Predictive Geometry of Hidden Trajectories in Transformers

arXiv:2609.37717v1 Announce Type: new Abstract: Decoder-only transformers are trained only through a terminal next-token prediction loss, yet this loss constrains every intermediate hidden state thro...

By Timur Mudarisov, Mikhail Burtsev, Tatiana Petrova, Radu State

llms efficiency

arXiv Machine Learning

Sep 30

Optimizer-dependent training dynamics converge to the same one-third optimal data scaling

arXiv:2609.37745v1 Announce Type: new Abstract: Neural scaling, in which loss falls as a power law with training, is central to large language models, and one recent proposal is that a $1/3$ exponent...

By Hyunseok Lee, Mihir Basil, Yizhou Liu, Jeff Gore

llms safety

arXiv AI

Sep 30

Privy to the Foil: Recasting Value Estimation with a Self-Privileged Critic for RLVR

arXiv:2609.37825v1 Announce Type: cross Abstract: Assigning credit to intermediate steps remains a central challenge in training Large Language Models (LLMs) on multi-step reasoning tasks with sparse...

By Kun Liang, Chenming Tang, Clive Bai, Weijie Liu, Zeyuan Liu, Qingyang Zhang, Saiyong Yang, Yunfang Wu

llms reinforcement-learning benchmarks

arXiv AI

Sep 30

Scaling Influence Functions in LLMs through Eigenbasis-Corrected One-Bit Gradient Projection

arXiv:2609.37842v1 Announce Type: cross Abstract: Influence functions estimate how individual training examples affect the behavior of large language models (LLMs). Analyzing how training data influe...

By Jaeseung Heo, J Rosser, Dongwoo Kim

llms fine-tuning efficiency

arXiv Machine Learning

Sep 30

Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning

arXiv:2609.37915v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) trains a student to match a privileged teacher distribution along its own sampled trajectory. Standard OPSD applies...

By Md. Ismail Hossain, Humaira Kousar, Isidora Chara Tourni

llms efficiency

arXiv Machine Learning

Sep 30

Pattern Formation in Transformers

arXiv:2609.37921v1 Announce Type: new Abstract: What are the inductive biases of a Transformer architecture? Existing theory on how the forward pass shapes representations either considers whether Tr...

By Erkan Turan, Gaspard Abel, Maks Ovsjanikov

llms

arXiv Machine Learning

Sep 30

TabFM-Auto: Self-Evolving Pipelines for Tabular Foundation Models

arXiv:2609.37989v1 Announce Type: new Abstract: Tabular foundation models achieve strong zero-shot accuracy on structured data by pretraining on synthetic tables, but they ignore the column names, ta...

By Deqing Fu, Huangyuan Su, Rajat Sen, Taman Narayan, Sujay Sanghavi, Abhimanyu Das, Weihao Kong

llms agents benchmarks data-engineering

arXiv Machine Learning

Sep 30

When do data mixtures improve scaling laws? Insights from high-dimensional regression

arXiv:2609.38011v1 Announce Type: new Abstract: Modern machine learning systems are trained on mixtures of data from different domains, and choosing the right mixture can substantially improve downst...

By Diyuan Wu, Lehan Chen, Theodor Misiakiewicz, Marco Mondelli

llms

arXiv AI

Sep 30

Dr. OPD: Learning What to Follow for Optimal On-Policy Distillation of Large Language Models

arXiv:2609.38025v1 Announce Type: cross Abstract: On-policy distillation (OPD) trains a student on its own generated responses using dense, token-level supervision from a stronger teacher. Vanilla OP...

By Zhenyu Wang, Tianze Wang, Linjun Zhang, Yifan Hu

llms efficiency

arXiv Machine Learning

Sep 30

Traversing the solution space of neural networks with Hessian Null Space Continuation

arXiv:2609.38081v1 Announce Type: new Abstract: On a single task, deep networks can learn many solutions, depending on their optimizer, training data, architecture, and hyperparameters. Many of these...

By Ann Huang, Mitchell Ostrow, Zhouyang Lu, William T. Redman, Leo Kozachkov, Kanaka Rajan

llms agents fine-tuning safety

arXiv Machine Learning

Sep 30

A Spectral Theory of Distortion in LLM Graph Reconstruction: Sharp Bounds and Empirical Characterization

arXiv:2609.38161v1 Announce Type: new Abstract: Evaluations of graph reconstruction by language models typically report a single aggregate distance between the original and the reconstructed graph. W...

By Jianru Shen

llms safety

arXiv Machine Learning

Sep 30

Sieve and Sage: Efficient Distraction Filtering for Reliable RALM Abstention

arXiv:2609.35794v1 Announce Type: cross Abstract: Just as Socrates recognized the limits of his own knowledge, Retrieval-Augmented Language Models (RALMs) should learn to abstain when the retrieved e...

By Jongbin Won, Sung Geun An, Jay-yoon Lee

llms rag safety

arXiv Machine Learning

Sep 30

Tracing mechanisms of sycophantic agreement in language models

arXiv:2609.35822v1 Announce Type: cross Abstract: Sycophantic agreement in language models refers to the tendency to overly affirm a user's stated beliefs or preferences, often at the expense of fact...

By Sixing Chen, Zhuofan Josh Ying, Logan Riggs Smith, Jeremy Wertheimer, Natalie Shapira

llms safety

arXiv AI

Sep 30

Same Bytes, Different Authority: Reserved-Token Representations in Chat-Template Prompt Injection

arXiv:2609.35932v1 Announce Type: cross Abstract: Prompt injection against LLM agents becomes much stronger when the injected instruction is wrapped in the model's own chat template. A forged templat...

By Yan Zhan, Yunze Song, Mengkai Hou, Wanting Zhang, Shaobo Liu, Zhijun Gao

llms rag agents nlp fine-tuning benchmarks

arXiv AI

Sep 30

Representation by Design in Generation: Cross-View Class-Token Alignment in Diffusion Transformers

arXiv:2609.36348v1 Announce Type: cross Abstract: Generative and representation learning remain asymmetrically connected: semantic representations are used to improve diffusion generation, whereas th...

By Xiaoyu Wu, Yifei Wang, Chen Wei

llms diffusion computer-vision safety

arXiv Machine Learning

Sep 30

Seeing What Should Be Heard: Diagnosing and Repairing Cross-Modal Shortcuts in Omni-Modal LLMs

arXiv:2609.36798v1 Announce Type: cross Abstract: Omni-modal large language models (LLMs) are expected to answer a question using the modality it explicitly refers to. However, existing training para...

By Yueran Ma, Ronghao Lin

llms nlp reinforcement-learning fine-tuning multimodal benchmarks

arXiv Machine Learning

Sep 30

You Cannot Recover What Was Never Measured: Quantifying the Information Ceiling of Ultra-Low-Field MRI Super-Resolution

arXiv:2609.36837v1 Announce Type: cross Abstract: Generative super-resolution models can turn portable 64 mT MRI into images that look like 3T scans, and the field evaluates them with PSNR, SSIM, and...

By Prathamesh Pradeep Khole, Shreya Handa, Utkarsh Gupta, Razvan Marinescu

llms diffusion benchmarks

arXiv AI

Sep 30

Safer Content or Firmer Refusals? A Hybrid Perturbation Defense for Alignment under Harmful Fine-tuning

arXiv:2609.36862v1 Announce Type: cross Abstract: Fine-tuning-as-a-service lets users adapt a safety-aligned language model to their own data, but it also creates a harmful fine-tuning attack surface...

By Muhammad Zeeshan Akram, Mufid Kamel Marican, Anvesh Reddy Yenugu, Ali Zain Kaimkhani, Minghong Fang

llms rag fine-tuning safety

arXiv Machine Learning

Sep 30

GleanVID: Complementary Token Selection for Efficient Video Large Language Models

arXiv:2609.37042v1 Announce Type: cross Abstract: Video Large Language Models (VideoLLMs) have achieved strong video understanding capabilities but incur substantial inference overhead due to the lar...

By Shuo Yang, Changbai Li, Rui Tang, Xinyu Zhao, Linlin Yang, Baochang Zhang

llms benchmarks

arXiv Machine Learning

Sep 30

vSkipper: Translating Dynamic Layer Skipping into LLM Serving Gains

arXiv:2609.37062v1 Announce Type: cross Abstract: Dynamic layer skipping reduces LLM computation by allowing each token to execute only a subset of the model's layers. However, existing skippers rely...

By Wei Da, Yavuz Ferhatosmanoglu, Evangelia Kalyvianaki

llms efficiency

Get the daily briefing