Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

27,837 stories · RSS feed

arXiv Machine Learning

Jun 18

Complementary Attention Head Pruning for Efficient Transformers

arXiv:2606. 19150v1 Announce Type: new Abstract: The remarkable success of Transformer-based models in natural language processing stems from architectural scaling, which leads to a large number of parameters and hinders deployment in resource-constrained environments.

By Yaniv Livertovsky, Shahar Somin, Gonen Singer

llms nlp efficiency benchmarks safety

arXiv AI

Jun 18

Beyond Reward Engineering: A Data Recipe for Long-Context Reinforcement Learning

arXiv:2606. 18831v1 Announce Type: cross Abstract: Long-context reasoning is an essential capability for large language models, particularly when they are deployed as autonomous agents that must reason over lengthy trajectories.

By Xiaoyue Xu, Sikui Zhang, Xiaorong Wang, Xu Han, Chaojun Xiao

llms agents reinforcement-learning benchmarks

arXiv AI

Jun 18

Caring Without Feeling: Affective Dynamics as the Control Layer of Human-AI Agent Collaboration

arXiv:2606. 18259v1 Announce Type: cross Abstract: AI agents that plan, retain memory across sessions, invoke external tools and act with partial autonomy are transforming human--AI collaboration.

By Junjie Xu, Xingjiao Wu, Zihao Zhang, Yujia Xu, Yuzhe Yang, Jin Zhu, Luwei Xiao, Wen Wu, Liang He

llms agents safety

arXiv AI

Jun 18

Reinforcement Learning Foundation Models Should Already Be A Thing

arXiv:2606. 18812v1 Announce Type: cross Abstract: Foundation models for language and vision are powered by internet-scale data, while structured domains (tabular prediction, time-series forecasting, graph learning, reinforcement learning) are not.

By Abdelrahman Zighem, Jill-J\^enn Vie

llms reinforcement-learning benchmarks

arXiv AI

Jun 18

User as Engram: Internalizing Per-User Memory as Local Parametric Edits

arXiv:2606. 19172v1 Announce Type: new Abstract: Personal memory in a language model is two problems: content and reasoning skill.

By Bojie Li

llms fine-tuning efficiency

arXiv AI

Jun 18

A Hybrid LSTM--Vision Transformer Architecture for Predicting HRRR Forecast Errors

arXiv:2606. 19026v1 Announce Type: cross Abstract: Forecast errors in high-resolution numerical weather prediction (NWP) systems are often linked to unresolved planetary boundary layer (PBL) processes, convection, terrain-induced circulations, and other vertically structured atmospheric phenomena.

By David Aaron Evans, Jay C. Rothenberger, Kara J. Sulia, Nick P. Bassill, Chris D. Thorncroft

llms computer-vision safety

arXiv AI

Jun 18

Graph Grounded Cross Attention Transformer Neural Network for Structurally Constrained Full Event Sequence Generation in Predictive Process Monitoring

arXiv:2606. 18726v1 Announce Type: cross Abstract: Structurally constrained event sequence generation remains challenging because generated paths must preserve transition feasibility, temporal order, termination, and attribute consistency.

By Fang Wang, Ernesto Damiani

llms benchmarks safety

arXiv AI

Jun 18

SAGE: Retain-Aware Post-Hoc Sanitization of Final Unlearning Vector

arXiv:2606. 18309v1 Announce Type: cross Abstract: Large Language Model (LLM) unlearning aims to remove undesirable knowledge or behaviors while preserving retained capabilities.

By Jingyuan Zhang, Yucheng Bai, Peixi Wen, Zhehao Huang, Zhengbao He, Hanling Tian, Xinwen Cheng, Haiyin Ran, Xiaolin Huang

llms benchmarks safety

arXiv AI

Jun 18

DRIFT: Refining Instruction Data via On-Policy Data Attribution

arXiv:2606. 18307v1 Announce Type: cross Abstract: Optimizing the training data distribution for Supervised Fine-Tuning (SFT) dictates the capability of Large Language Models (LLMs).

By Zefan Wang, Lincheng Li, Tianyu Yu, Yuan Yao

llms fine-tuning safety

arXiv AI

Jun 18

BCL: Bayesian In-Context Learning Framework for Information Extraction

arXiv:2606. 18620v1 Announce Type: cross Abstract: Existing information extraction (IE) tasks increasingly adopt in-context learning (ICL) with large language models.

By Haoliang Liu, Chengkun Cai, Xu Zhao, Han Zhu, Shizhou Huang, Xinglin Zhang, Tao Chen, Jenq-Neng Hwang, Zhang Huaping, Lei Li

llms

arXiv AI

Jun 18

Trade-offs in Medical LLM Adaptation: An Empirical Study in French QA

arXiv:2606. 19266v1 Announce Type: cross Abstract: The development of large language models (LLMs) has led to an increased focus on their adaptation to specialized domains and languages, yet the effectiveness of domain adaptation strategies remains unclear.

By Ikram Belmadani, Oumaima El Khettari, Carlos Ramisch, Frederic Bechet, Richard Dufour, Benoit Favre

llms fine-tuning benchmarks

arXiv AI

Jun 18

Surrogate Benchmarks for Model Merging Optimization

arXiv:2509. 02555v2 Announce Type: replace-cross Abstract: Model merging techniques aim to integrate the abilities of multiple models into a single model.

By Rio Akizuki, Yuya Kudo, Nozomu Yoshinari, Yoichi Hirose, Toshiyuki Nishimoto, Kento Uchida, Shinichi Shirakawa

llms benchmarks

arXiv AI

Jun 18

RTSGameBench: An RTS Benchmark for Strategic Reasoning by Vision-Language Models

arXiv:2606. 18950v1 Announce Type: new Abstract: Modern Vision-Language Models (VLMs) often struggle with strategic reasoning, i.

By San Kim, Daechul Ahn, Reokyoung Kim, Hyeonbeom Choi, Seungyeon Jwa, Jonghyun Choi

llms agents multimodal benchmarks

arXiv AI

Jun 18

Rescaling MLM-Head for Neural Sparse Retrieval

arXiv:2606. 18811v1 Announce Type: cross Abstract: Learned sparse retrieval (LSR) models such as SPLADE have traditionally used BERT-style masked language models as backbone encoders.

By Youngjoon Jang, Seongtae Hong, Jonah Turner, Heuiseok Lim

llms benchmarks

arXiv AI

Jun 18

Beyond Similarity: Temporal Operator Attention for Time Series Analysis

arXiv:2605. 11287v2 Announce Type: replace-cross Abstract: A persistent paradox in time-series forecasting is that structurally simple MLP and linear models often outperform high-capacity Transformers.

By Jevon Twitty, Vinh Pham, Nitiwith Rotchanarak, Viresh Pati, Yubin Kim, Shihao Yang, Jiecheng Lu

llms benchmarks

arXiv AI

Jun 18

AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice

arXiv:2606. 18617v1 Announce Type: cross Abstract: There exist numerous tutor training platforms.

By Danielle R. Thomas, Marie Cynthia Abijuru Kamikazi, Clara Brandt, Conrad Borchers, Kenneth R. Koedinger

llms

arXiv Machine Learning

Jun 18

Contextualizing Biological Language Models across Modalities via Logit-Space Contrastive Alignment

arXiv:2606. 18703v1 Announce Type: new Abstract: Pretrained biological language models expose per-token probability distributions through masked-token prediction, providing the likelihood interface central to sequence design, variant scoring, and mechanistic interpretation.

By Yanjun Shao, Yundi Chen, Yashvi Patel, Aurelien Pelissier, Mar\'ia Rodr\'iguez Mart\'inez

llms rag nlp multimodal benchmarks safety

arXiv AI

Jun 18

A Link between Shock-wave Theory and Symmetry-reduced Stochastic Gradient Descent for Artificial Neural Networks

arXiv:2606. 18303v1 Announce Type: cross Abstract: We develop a mathematically explicit link between shock-wave theory and the symmetry-quotiented learning dynamics of stochastic gradient descent, drawing on differential geometry, Lie group theory, and fluid mechanics.

By Taiki Miyagawa

llms

arXiv AI

Jun 18

Generating Natural and Expressive Robot Gestures through Iterative Reinforcement Learning with Human Feedback using LLMs

arXiv:2606. 18747v1 Announce Type: cross Abstract: Expressive gestures are essential for natural and effective communication, complementing speech when verbal cues alone are insufficient (e.

By Chris Lee, Flora Salim, Benjamin Tag, Francisco Cruz

llms reinforcement-learning robotics

arXiv AI

Jun 18

Steerable Cultural Preference Optimization of Reward Models

arXiv:2606. 18606v1 Announce Type: cross Abstract: It is essential for large language model (LLM) technology to serve many different cultural sub-communities in a manner that is acceptable to each community.

By Minsik Oh, Advit Deepak, Sophie Wu, Douwe Kiela, Ekaterina Shutova

llms reinforcement-learning fine-tuning safety

Get the daily briefing