Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

19,764 stories · RSS feed

arXiv Machine Learning

Jul 21

HyBDM: Multi-Scale Hybrid Experts for Time Series Forecasting with Bidirectional Dependency Modeling

arXiv:2607. 16882v1 Announce Type: new Abstract: Time series forecasting (TSF) is vital to many applications, yet existing models often struggle to capture the heterogeneous long-range global patterns and short-range local variations in multivariate time series.

By Wenqiang Ma, Chen Cheng, Xue Cheng, Jiarui Ye

llms benchmarks

arXiv Machine Learning

Jul 21

Node4All: Learning Node Representation Beyond Datasets

arXiv:2607. 17272v1 Announce Type: new Abstract: Node representation learning has advanced rapidly, yet most existing methods rely on per-dataset training and hyperparameter tuning.

By Dooho Lee, Jaemin Yoo

llms benchmarks

arXiv Machine Learning

Jul 21

AGG: Jacobian-Aggregated Group Gradient for Efficient GRPO Training of Diffusion Models

arXiv:2607. 17572v1 Announce Type: new Abstract: Group Relative Policy Optimization (GRPO) is a powerful reinforcement learning algorithm for aligning generative models with human preferences.

By Ruiyi Ding, Jie Li, He Kang, Ziyan Liu, Chengru Song, Yuan chen

llms diffusion reinforcement-learning benchmarks

arXiv Machine Learning

Jul 21

Feature-Guided Diffusion for Non-Differentiable Inverse Rendering

arXiv:2607. 17411v1 Announce Type: cross Abstract: Inverse rendering is traditionally solved via differentiable renderers and gradient descent, which requires substantial problem-specific engineering and is prone to getting stuck in local minima due to ambiguities.

By Andrei-Timotei Ardelean, Michael Fischer, Tim Weyrich, Tom\'a\v{s} Iser

llms diffusion computer-vision robotics

arXiv AI

Jul 21

GenSyn10: A Multi-Generative AI Dataset For Benchmarking Image Classification

arXiv:2607. 16283v1 Announce Type: cross Abstract: The rapid advancement of generative AI has outpaced our ability to reliably detect its outputs, particularly when detectors encounter generators they have not seen before.

By Md Faraz Kabir Khan, Saeed Anwar, Ghulam Mubashar Hassan

llms diffusion computer-vision fine-tuning multimodal benchmarks

arXiv Machine Learning

Jul 21

An Adjoint-Sensitivity Framework for Lost-in-the-Middle Phenomena in Causal Residual Transformers

arXiv:2607. 17696v1 Announce Type: cross Abstract: We develop an adjoint-sensitivity framework for positional influence in causal residual Transformers and separate unconditional analytic results from conditional boundary-shape conclusions.

By Cheng Huan, Hongwei Yuan

llms safety

arXiv Machine Learning

Jul 21

Patch Policy: Efficient Embodied Control via Dense Visual Representations

arXiv:2607. 18236v1 Announce Type: cross Abstract: Pretrained dense visual features from Vision Transformers (ViTs) are powerful yet have been underutilized in robot learning.

By Gaoyue Zhou, Zichen Jeff Cui, Ada Langford, Bowen Tan, Yann LeCun, Lerrel Pinto

llms robotics fine-tuning efficiency multimodal benchmarks

arXiv Machine Learning

Jul 21

NIRVANA: Structured Pruning Reimagined for Large Language Model Compression

arXiv:2509. 14230v2 Announce Type: replace Abstract: While structured pruning presents a highly effective pathway for accelerating Large Language Model (LLM) inference, existing methods frequently suffer from significant performance degradation and demand computationally retraining to recover capabilities.

By Mengting Ai, Tianxin Wei, Sirui Chen, Jingrui He

llms fine-tuning efficiency benchmarks

arXiv AI

Jul 21

Can AI Agents Really Complete RTL-to-GDS? Lessons from Benchmarking Tool-Interactive EDA Workflows

arXiv:2607. 17528v1 Announce Type: new Abstract: LLM-driven agent systems have emerged as a promising paradigm for electronic design automation (EDA), demonstrating strong potential for automating complex design workflows.

By Jinyuan Deng, Zhengrui Chen, Xufeng Wei, Tianyu Xing, Chenyi Wen, Cheng Zhuo

llms agents benchmarks

arXiv Machine Learning

Jul 21

Sketching the Readout of Large Language Models for Scalable Data Attribution and Valuation

arXiv:2604. 16197v2 Announce Type: replace Abstract: Data attribution and valuation are critical for understanding data-model synergy for Large Language Models (LLMs), yet existing gradient-based methods suffer from scalability challenges on LLMs.

By Yide Ran, Jianwen Xie, Minghui Wang, Wenjin Zheng, Denghui Zhang, Chuan Li, Zhaozhuo Xu

llms

arXiv Machine Learning

Jul 21

OpenLanguageModel: Readable and Composable Small-Language-Model Pretraining for Education and Research

arXiv:2607. 16669v1 Announce Type: cross Abstract: OpenLanguageModel (OLM) is an open-source PyTorch library for building and pretraining small language models while keeping their machinery visible.

By Tavish Mankash, Vardhaman Kalloli, Keshava Prasad, Deepan Muthirayan

llms benchmarks

arXiv Machine Learning

Jul 21

DORA: A Scalable Asynchronous Reinforcement Learning System for Language Model Training

arXiv:2604. 26256v2 Announce Type: replace Abstract: Reinforcement learning (RL) has become a critical paradigm for LLM post-training, yet the rollout phase -- accounting for 50--80% of total step time -- is bottlenecked by skewed generation: long-tailed trajectories indispensable for model performance block the entire training pipeline.

By Tianhao Hu, Xiangcheng Liu, Yuchun Miao, Youshao Xiao, Hongyu Zang, Yang Zheng, Xuan Huang, Jinrui Ding, Yufei Zhang, Yu Yang, Yi-Kai Zhang, Yueqing Sun, Chengcheng Han, Xiandi Ma, Wei Wang, Qi Gu, Yerui Sun, Yuchen Xie, Xunliang Cai

llms agents reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Jul 21

Quantifying Ranking Uncertainty in LLM Benchmarks

arXiv:2607. 16259v1 Announce Type: new Abstract: Pretrained models are typically ranked on multi-task leaderboards to assess their effectiveness across diverse tasks.

By Bitya Neuhof, Yuval Benjamini

llms benchmarks

arXiv Machine Learning

Jul 21

Gradient-Free Privacy Leakage in Federated Language Models through Selective Weight Tampering

arXiv:2310. 16152v5 Announce Type: replace-cross Abstract: Federated learning (FL) has become a key component in various language modeling applications such as machine translation, next-word prediction, and medical record analysis.

By Md Rafi Ur Rashid, Vishnu Asutosh Dasu, Kang Gu, Najrin Sultana, Shagufta Mehnaz

llms nlp safety

arXiv AI

Jul 21

Phantom Transitions in Language Model Fine-Tuning: A Density-Matrix Analysis

arXiv:2606. 07559v2 Announce Type: replace-cross Abstract: Fine-tuning a language model often fails silently when its correct completion must outrank a near-synonym competitor.

By Vaibhav Prakash, Jayasri Dontabhaktuni

llms rag fine-tuning

arXiv Machine Learning

Jul 21

Singular Bayesian Neural Networks

arXiv:2602. 00387v4 Announce Type: replace-cross Abstract: Bayesian neural networks promise calibrated uncertainty but require $O(mn)$ parameters for standard mean-field Gaussian posteriors.

By Mame Diarra Toure, David A. Stephens

llms benchmarks safety

arXiv Machine Learning

Jul 21

FAME: Failure-Aware Mixture-of-Experts for Message-Level Log Anomaly Detection

arXiv:2605. 22779v2 Announce Type: replace-cross Abstract: Production systems generate millions of log lines daily, yet most anomaly detectors operate at the session or window-level, flagging groups of lines rather than identifying the specific message responsible.

By Huanchi Wang, Zihang Huang, Yifang Tian, Kristina Dzeparoska, Hans-Arno Jacobsen, Alberto Leon-Garcia

llms

arXiv Machine Learning

Jul 21

SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks?

arXiv:2605. 26548v2 Announce Type: replace-cross Abstract: Finding a real vulnerability in complicated systems is a challenging, long-horizon task that demands reasoning across an entire codebase to produce a working proof-of-concept (PoC).

By Hwiwon Lee, Jiawei Liu, Dongjun Kim, Wubing Xia, Ziqi Zhang, Chunqiu Steven Xia, Lingming Zhang

llms agents benchmarks safety

arXiv AI

Jul 21

Otap:Structure-Aware Optimal Transport for Evaluating Planning and Execution in Agent Trajectories

arXiv:2607. 17082v1 Announce Type: new Abstract: Large language model agents solve tasks by generating trajectories that interleave planning, tool calls, and intermediate results.

By Babak Barazandeh, Subhabrata Majumdar, George Michailidis

llms agents benchmarks

arXiv AI

Jul 21

CoCurve: Cross-Module Co-Pruning Curvature for Training-Free Structured LLM Pruning

arXiv:2607. 17568v1 Announce Type: cross Abstract: Structured pruning compresses large language models (LLMs) by removing whole computational units, such as attention heads and feed-forward (FFN) channel groups.

By Zhiren Gong, Zihao Zeng, Zijie Wang, Tiantong Wang, Chau Yuen, Wei Yang Bryan Lim

llms fine-tuning efficiency benchmarks

Get the daily briefing