Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

29,760 stories · RSS feed

arXiv AI

Jun 9

Pruning and Distilling Mixture-of-Experts into Dense Language Models

arXiv:2605. 28207v2 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) is now the dominant architecture for frontier language models, yet it requires all expert parameters to be loaded in memory, making it less preferable for memory-constrained deployment.

By Junhyuck Kim, Jihun Yun, Haechan Kim, Gyeongman Kim, Joonghyun Bae, Jaewoong Cho

llms efficiency

arXiv AI

Jun 9

Evaluating Hallucinations in Domain-Adapted Large Language Models

arXiv:2606. 07521v1 Announce Type: cross Abstract: This study investigates the phenomenon of hallucinations in domain-adapted Large Language Models (LLMs), focusing on the fine-tuning of the Llama-2 model with the Lamini dataset.

By Sanchita Porwal, Sai Prasath S, Xingjian Bi, Madelyn Scandlen

llms fine-tuning safety

arXiv AI

Jun 9

CrossVLA: Cross-Paradigm Post-Training and Inference Optimization for Vision-Language-Action Models

arXiv:2605. 21854v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models have rapidly converged on a small set of architectural patterns: discrete-token autoregression (e.

By Zhi Liu

llms reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv AI

Jun 9

Post-training is (Massive) Supervised Learning

arXiv:2606. 07527v1 Announce Type: cross Abstract: The prevailing paradigm for training LLMs has evolved to rely on a massive post-training phase consisting of SFT and RL.

By Michael Hassid, Yossi Adi, Roy Schwartz

llms reinforcement-learning fine-tuning benchmarks

arXiv Machine Learning

Jun 9

AutoMegaKernel: A Statically-Checked Agent Harness for Self-Retargeting Megakernel Synthesis

arXiv:2606. 09682v1 Announce Type: new Abstract: AutoMegaKernel (AMK) compiles a HuggingFace Llama-family model into a single persistent cooperative CUDA kernel that runs the whole forward pass in one launch, with no per-model hand-written CUDA.

By Jaber Jaber, Osama Jaber

llms agents safety

arXiv Machine Learning

Jun 9

IGenBench: Benchmarking the Reliability of Text-to-Infographic Generation

arXiv:2601. 04498v2 Announce Type: replace Abstract: Infographics are composite visual artifacts that combine data visualizations with textual and illustrative elements to communicate information.

By Yinghao Tang, Xueding Liu, Boyuan Zhang, Tingfeng Lan, Yupeng Xie, Jiale Lao, Yiyao Wang, Haoxuan Li, Tingting Gao, Bo Pan, Luoxuan Weng, Xiuqi Huang, Minfeng Zhu, Yingchaojie Feng, Yuyu Luo, Wei Chen

llms diffusion multimodal benchmarks

arXiv Machine Learning

Jun 9

The Sample Complexity of Parameter-Free Stochastic Convex Optimization

arXiv:2506. 11336v2 Announce Type: replace Abstract: We study the sample complexity of stochastic convex optimization when problem parameters such as the distance to optimality and the Lipschitz constant are unknown.

By Jared Lawrence, Ari Kalinsky, Hannah Bradfield, Yair Carmon, Oliver Hinder

llms fine-tuning

arXiv AI

Jun 9

When Does Delegation Beat Majority? A Delegation-Based Aggregator for Multi-Sample LLM Inference

arXiv:2606. 08098v1 Announce Type: new Abstract: Majority voting over sampled answers is the dominant unsupervised aggregator for multi-sample LLM inference.

By Yasushi Sakai, Allen Song, Kent Larson

llms rag benchmarks

arXiv AI

Jun 9

MatSciBench: Benchmarking the Reasoning Ability of Large Language Models in Materials Science

arXiv:2510. 12171v2 Announce Type: replace Abstract: Large Language Models have shown strong scientific reasoning ability, but their performance on materials science problems remains less studied.

By Junkai Zhang, Jingru Gan, Xiaoxuan Wang, Zian Jia, Changquan Gu, Jianpeng Chen, Yanqiao Zhu, Mingyu Derek Ma, Dawei Zhou, Ling Li, Wei Wang

llms multimodal benchmarks

arXiv AI

Jun 9

Larch: Learned Query Optimization for Semantic Predicates

arXiv:2606. 07923v1 Announce Type: cross Abstract: With the advent of Large Language Models (LLMs), many database systems introduced semantic operators that enabled analytical queries over unstructured data (e.

By Fuheng Zhao, Pawel Liskowski, Zihan Li, Benjamin Han, Puxuan Yu, Varich Boonsanong, Dimitris Tsirogiannis, Anupam Datta

llms rag reinforcement-learning data-engineering

arXiv AI

Jun 9

Learning Task Mixtures from Task Affinities: A Probabilistic Graphical Model for Supervised Fine-Tuning

arXiv:2507. 12612v4 Announce Type: replace-cross Abstract: Supervised fine-tuning performance for large language models depends strongly on how training budget is distributed across a heterogeneous set of tasks.

By Prateek Chanda, Saral Sureka, Parth Pratim Chatterjee, Krishnateja Killamsetty, Nikhil Shivakumar Nayak, Ganesh Ramakrishnan

llms fine-tuning

arXiv AI

Jun 9

Cost-Aware Speculative Execution for LLM-Agent Workflows: An Integrated Five-Dimension Method

arXiv:2606. 07846v1 Announce Type: cross Abstract: LLM-agent workflows chain model calls and tool invocations, and spend most of their wall-clock time waiting on upstream operations before downstream ones can start.

By Faisal Fareed

llms agents

arXiv AI

Jun 9

PLAGUE: Plug-and-play framework for Lifelong Adaptive Generation of Multi-turn Exploits

arXiv:2510. 17947v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are improving at an exceptional rate.

By Neeladri Bhuiya, Madhav Aggarwal, Diptanshu Purwar

llms agents multimodal benchmarks safety

arXiv AI

Jun 9

mllm-shap: A Shapley Value Explainability Platform for Text-Audio Multimodal Large Language Models

arXiv:2606. 07531v1 Announce Type: cross Abstract: We introduce mllm-shap, an open-source Python framework designed to extend Shapley Value (SV) explainability from text-only Large Language Models to Multimodal LLMs (MLLMs) processing joint text and audio inputs.

By Jakub Muszy\'nski, Pawe{\l} Pozorski, Maria Ganzha

llms multimodal safety

arXiv AI

Jun 9

Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps

arXiv:2605. 16928v2 Announce Type: replace-cross Abstract: Long-context inference in large language models is bottlenecked by the quadratic cost of full attention.

By Yanke Zhou, Yiduo Li, Hanlin Tang, Maohua Li, Kan Liu, Tao Lan, Lin Qu, Yuan Yao, Xiaoxing Ma

llms efficiency benchmarks

arXiv Machine Learning

Jun 9

Breaking the Tokenizer Barrier: On-Policy Distillation across Model Families

arXiv:2606. 09456v1 Announce Type: new Abstract: On-Policy Distillation (OPD) has become a core technique in the post-training of Large Language Models (LLMs) for transferring knowledge from domain experts to student models.

By Yifan Niu, Han Xiao, Dongyi Liu, Zelong Wang, Dihong Gong, Yasheng Wang, Jia Li

llms nlp fine-tuning efficiency benchmarks

arXiv Machine Learning

Jun 9

In-Context Learning of Stochastic Differential Equations with Foundation Inference Models

arXiv:2502. 19049v3 Announce Type: replace Abstract: Stochastic differential equations (SDEs) describe dynamical systems where deterministic flows, governed by a drift function, are superimposed with random fluctuations, dictated by a diffusion function.

By Patrick Seifner, Kostadin Cvejoski, David Berghaus, Cesar Ojeda, Ramses J. Sanchez

llms diffusion fine-tuning

arXiv AI

Jun 9

WhiFlash: Accelerating Speculative Decoding with Token-Level Cross-Paradigm Routing

arXiv:2606. 07710v1 Announce Type: cross Abstract: The autoregressive nature of large language models (LLMs) remains a significant bottleneck for inference, particularly in complex agentic workloads.

By Young D. Kwon, Miles Williams, Rui Li, Alexandros Kouris, Stylianos I. Venieris

llms agents diffusion efficiency benchmarks

arXiv AI

Jun 9

Towards Long-Horizon Vessel Trajectory and Destination Forecasting with Reasoning Large Language Models

arXiv:2606. 08633v1 Announce Type: new Abstract: Long-horizon maritime trajectory prediction is important for shipping management, logistics planning, and maritime risk analysis, yet month-level forecasting remains insufficiently studied.

By Hongwei Wang, Miao Zhou, Fengde Wang, Yuting Wang, Jiewen Yu, Jun-Yan He, Bohao Qu, Wanbing Zhang, Xiuju Fu, Qing Guo, Zipei Fan, Yingying Xing, Yi Yuan

llms reinforcement-learning fine-tuning benchmarks

arXiv AI

Jun 9

A Survey on Large Language Model-Based Game Agents

arXiv:2404. 02039v5 Announce Type: replace Abstract: Game environments provide rich, controllable settings that stimulate many aspects of real-world complexity.

By Sihao Hu, Tiansheng Huang, Gaowen Liu, Ramana Rao Kompella, Fatih Ilhan, Selim Furkan Tekin, Yichang Xu, Zachary Yahn, Ling Liu

llms agents

Get the daily briefing