Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

28,678 stories · RSS feed

arXiv AI

Oct 2

Talked Out of the Truth: Sycophancy in the Reasoning Chains of Multimodal Models

arXiv:2608.28623v2 Announce Type: replace-cross Abstract: Large multimodal reasoning models (LMRMs) are increasingly capable, largely through generating explicit chain-of-thought reasoning before ans...

By Mahir Numayeer Islam, Gakuto Okuyama, Nikolaus Siauw, Shivank Garg, Madhur Panwar, Vasu Sharma

llms multimodal benchmarks

arXiv Machine Learning

Oct 2

Role-guided Speaker Deletion Verification in Clinical Psychiatry Speech Recordings with Audio Language Models

arXiv:2609.38491v1 Announce Type: new Abstract: Clinical research in psychiatry increasingly relies on large scale collection of spoken language data to identify acoustic and linguistic biomarkers. Y...

By Joseph T Colonel, Daniel Katzman, Kelsey Kirker, Adam N Davidson, Shalaila S Haas, Cheryl Corcoran, Ren\'{e} S Kahn, Guillermo Checci, Baihan Lin

llms

arXiv Machine Learning

Oct 2

Lasting Effects of Abstract Pretraining Beyond Perplexity

arXiv:2609.38764v1 Announce Type: new Abstract: Language models are typically pretrained from random initialization. Recent work challenges this convention, showing that a brief warm-up on abstract,...

By Zachary Shinnick, Hemanth Saratchandran, Damien Teney, Anton van den Hengel

llms nlp robotics

arXiv Machine Learning

Oct 2

SCORE-LM: State-Space Radar Representations with Language Models for Fault Diagnosis

arXiv:2609.38980v1 Announce Type: new Abstract: Radar hardware faults threaten automated perception, motivating accurate, compact diagnosis and understandable maintenance guidance. We introduce SCORE...

By Mainak Mallick, Seung-Kyum Choi

llms benchmarks

arXiv Machine Learning

Oct 2

SparseEngine: Sparse-First Inference Engine

arXiv:2609.39068v1 Announce Type: new Abstract: Long-context LLM agents accumulate interaction histories that strain KV-cache memory and attention computation. Although sparse attention reduces these...

By Jitai Hao, Quansheng Gu, Qiang Huang, Jun Yu

llms agents efficiency benchmarks

arXiv Machine Learning

Oct 2

The Row Normalization Puzzle in Muon

arXiv:2609.39114v1 Announce Type: new Abstract: This paper examines how row-wise renormalization affects Muon, focusing on the gap between NorMuon's worst-case guarantees and its practical performanc...

By Jiayu Zhang, Tianyi Lin

llms

arXiv Machine Learning

Oct 2

Characterizing High Bandwidth Flash for LLM Serving

arXiv:2609.39131v1 Announce Type: new Abstract: Large language model (LLM) serving requires substantial memory to store model weights and KV caches. As models grow larger and contexts become longer,...

By Zack Yu, Chloe Wong, Coleman Hooper, Minjae Lee, Wonjun Kang, Youngjin Cho, Michael W. Mahoney, Yakun Sophia Shao, Kurt Keutzer, Amir Gholami

llms agents efficiency

arXiv Machine Learning

Oct 2

Attention Function as an Intrinsic Inductive Bias: How Models' Behavior Diverges in Novel Contexts

arXiv:2609.39188v1 Announce Type: new Abstract: Developmental psychology holds that certain priors are given to infants prior to experience rather than induced from data, and that the influence of su...

By Dong Gyun Kang, Megha Thukral, Kwangsoo Kim

llms safety

arXiv Machine Learning

Oct 2

QATFactory: A Versatile, Deployment-Aligned Framework for Quantization-aware Training and Distillation of LLMs

arXiv:2609.39223v2 Announce Type: new Abstract: Large language model (LLM) inference is increasingly moving toward lower precision to realize the throughput of hardware accelerators, but aggressive p...

By Weili Xu, Jisen Li, Yuqing Jian, Chenxi Li, Zhizhou Sha, Yifan Yu, Qingyang Wu, Chenfeng Xu, Zhongzhu Zhou, Tianyi Zhang, Ben Athiwaratkun

llms reinforcement-learning fine-tuning efficiency benchmarks

arXiv Machine Learning

Oct 2

Right Answer, Wrong Mechanism: Detecting Pernicious Divergence in Causal Interventions

arXiv:2609.39243v1 Announce Type: new Abstract: Causal interventions such as activation patching and distributed alignment search (DAS) are the main tool for making mechanistic claims about neural ne...

By Beiming Liu, Minjie Chen

llms benchmarks safety

arXiv Machine Learning

Oct 2

PatchKV: Weight-Space Compensation of KV Cache

arXiv:2609.39329v1 Announce Type: new Abstract: Long-context inference with Large Language Models (LLMs) is bottlenecked by the linearly growing memory of the key-value (KV) cache. Existing compressi...

By Chanryeol Lee, Chanhyuk Lee, Yeonwoo Choi, Donggyun Kim, Seunghoon Hong

llms efficiency benchmarks

arXiv Machine Learning

Oct 2

CDMD: A Cross-Dataset Mixed-Type Diffusion Model for Tabular Data

arXiv:2609.39124v1 Announce Type: new Abstract: Generative models for tabular data are typically trained separately for each dataset, limiting knowledge transfer and requiring the storage of many spe...

By Mohamed Amine Ketata, Maximilian Schambach, Stephan G\"unnemann

llms diffusion

arXiv Machine Learning

Oct 2

Learning Reliable GUI Agents under Imperfect Priors

arXiv:2609.39547v1 Announce Type: new Abstract: GUI agents built on large language and vision-language models still struggle on unseen applications and complex multi-step tasks, as completing real GU...

By Bo Han, Qianyi Wang, Shuai Liu, Xiong Zifan, Changqiao Wu, Yuanfa Li, Pengzhi Gao, Wei Liu, Jian Luan, Heng Qu, Yunpeng Song, Zhongmin Cai

llms rag agents multimodal benchmarks

arXiv Machine Learning

Oct 2

Self-Repulsive Sampling for Diffusion Language Models

arXiv:2609.39560v1 Announce Type: new Abstract: Sampling several responses and voting over their answers can improve a language model's accuracy, but repeated answers limit the benefit of additional...

By Michael Helcig, Martin Jaggi

llms diffusion benchmarks

arXiv Machine Learning

Oct 2

A library for differentiable signal processing and machine learning on the sphere

arXiv:2609.39737v1 Announce Type: new Abstract: The two-dimensional sphere embedded in three-dimensional Euclidean space S2, plays a central role in a variety of scientific and engineering domains, i...

By Thorsten Kurth, Max Rietmann, Mauro Bisson, Andrea Paris, Alberto Carpentieri, Jean Kossaifi, Anima Anandkumar, Christian Hundt, Boris Bonev

llms benchmarks

arXiv Machine Learning

Oct 2

Stable Transformers for Graph Generation

arXiv:2609.39739v1 Announce Type: new Abstract: Graph generative models increasingly rely on Graph Transformers (GT) to capture complex dependencies among nodes and edges. While deeper architectures...

By Luca Miglior, Alessio Gravina, Davide Bacciu

llms benchmarks

arXiv Machine Learning

Oct 2

GraphMAS: A Systematic Benchmark of Multi-Agent Coordination for Graph Learning

arXiv:2609.39777v1 Announce Type: new Abstract: LLM-based multi-agent systems coordinate specialized reasoning through aggregation, interaction, and adaptive control, yet their potential for graph le...

By Jiayi Yang, Yifang Chen, Yuanfu Sun, Xinyan Ge, Qiaoyu Tan

llms agents benchmarks

arXiv Machine Learning

Oct 2

Beyond Accuracy: Prefix-Invariant Realizations of Low-Precision Fast Matrix Multiplication

arXiv:2609.39816v1 Announce Type: new Abstract: Fast matrix multiplication saves multiplications through exact cancellation, but rounding sums that mix token rows can leave contributions from later t...

By Shuxiao Xie, Shuyang Xie, Yuan Cao, Dezhi Ran, Wei Yang, Tao Xie

llms efficiency

arXiv Machine Learning

Oct 2

Replay on Demand: An Emergent Curriculum for Balancing Adaptation and Forgetting in Continued Pretraining

arXiv:2609.40089v1 Announce Type: new Abstract: Continued pretraining enables language models to adapt to new domains and knowledge, but often at the cost of forgetting previously acquired capabiliti...

By Lukas Thede, Shengzhuang Chen, Stefan Winzeck, Matthias Bethge, Zeynep Akata, Jonathan Richard Schwarz

llms

arXiv Machine Learning

Oct 2

Is Weight Tying Still Beneficial for Decoder-Only LLMs in Private Settings Under DP-SGD?

arXiv:2609.40335v1 Announce Type: new Abstract: Differentially Private Stochastic Gradient Descent (DP-SGD) is a leading approach for privacy-preserving fine-tuning of large language models (LLMs). M...

By Razan El Mais, Ali Chehab, Ibrahim Issa, Razane Tajeddine

llms rag fine-tuning safety

Get the daily briefing