Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

26,473 stories · RSS feed

arXiv Machine Learning

Jun 25

ASAP: Agent-System Co-Design for Wall-Clock-Centered Auto HPO Research for ML Experiments

arXiv:2606. 25207v1 Announce Type: new Abstract: Hyperparameter Optimization (HPO) is essential for maximizing machine learning model performance, and its core challenge is sample efficiency: finding strong configurations within a limited budget.

By Taicheng Guo, Haomin Zhuang, Kehan Guo, Yujun Zhou, Nitesh V. Chawla, Olaf Wiest, Xiangliang Zhang

llms agents safety

arXiv Machine Learning

Jun 25

Natural Ungrokking: Asymmetric Control of Which Rules Survive Pretraining

arXiv:2606. 26050v1 Announce Type: new Abstract: Midway through an ordinary pretraining run, a small language model learns the pronoun-gender rule: cued with a girl's name ("Sue cried because"), it resolves the next pronoun to she, generalizing to held-out probes (0.

By Juliana Li, Diya Sreedhar

llms

arXiv Machine Learning

Jun 25

Closed-Loop Graph Algorithm Execution with Small Language Models: Step Accuracy and Rollout Reliability

arXiv:2606. 24980v1 Announce Type: new Abstract: Small language models offer an efficient alternative to large-scale systems, but their ability to execute structured algorithms over multiple dependent decisions remains poorly understood.

By Michal Podstawski

llms agents

arXiv Machine Learning

Jun 25

Curvature-Guided Mixing for MLLM Adaptation

arXiv:2606. 24963v1 Announce Type: cross Abstract: Fine-tuning Multimodal Large Language Models (MLLMs) on specialized tasks often leads to catastrophic forgetting of their general capabilities.

By Jinglong Yang, Jiaxuan He, Wenjian Huang, Zhan Zhuang, Jianguo Zhang

llms fine-tuning multimodal

arXiv Machine Learning

Jun 25

Geo-Strat-RL: Learning Geological Event Reasoning from Verifiable Tasks

arXiv:2606. 25000v1 Announce Type: new Abstract: To evaluate whether vision-language models can reason about geological histories, it is necessary to construct observations for which the underlying process history is known.

By Lukas Mosser

llms reinforcement-learning multimodal

arXiv Machine Learning

Jun 25

Exposing the Illusion of Erasure in Knowledge Editing for LLMs

arXiv:2606. 23276v2 Announce Type: replace Abstract: Knowledge Editing (KE) has emerged as a frontier for updating specific facts in LLMs without costly retraining, but its reliability and underlying mechanisms remain poorly understood.

By Advik Raj Basani, Anshuman Chhabra

llms safety

arXiv Machine Learning

Jun 25

AeroCast: Probabilistic 3D Trajectory Prediction for Non-Cooperative Aerial Obstacles via Transformer-MDN Architecture

arXiv:2606. 25122v1 Announce Type: cross Abstract: Autonomous aerial vehicles operating in shared airspace must predict the future positions of non-cooperative obstacles to plan evasive maneuvers before a collision becomes unavoidable.

By Syed Izzat Ullah, Jose Baca

llms agents multimodal benchmarks

arXiv Machine Learning

Jun 25

InvestPhilBench: A Multi-Layer Dynamic Benchmark for Evaluating Large Language Model Procedural Reasoning in Expert Investment Philosophy

arXiv:2606. 25984v1 Announce Type: cross Abstract: Large language models are increasingly deployed as investment research assistants, yet no benchmark tests whether they can accurately reconstruct and apply the specific procedural decision frameworks of expert investors.

By Mingguang Chen, Bo Qu

llms benchmarks

arXiv Machine Learning

Jun 25

MINIF2F-DAFNY: LLM-Guided Mathematical Theorem Proving via Auto-Active Verification

arXiv:2512. 10187v3 Announce Type: replace Abstract: LLMs excel at reasoning, but validating their steps remains challenging.

By Mantas Baksys, Stefan Zetzsche, Olivier Bouissou, Sean B. Holden

llms benchmarks

arXiv Machine Learning

Jun 25

Memory Contagion: Cross-Temporal Propagation of Evaluator Bias via Agent Memory

arXiv:2606. 23195v2 Announce Type: replace Abstract: Large Language Model (LLM) agents increasingly rely on memory systems to maintain long-term coherence.

By Zewen Liu

llms agents safety

arXiv Machine Learning

Jun 25

Communicability-Inspired Positional Encoding (CIPE)

arXiv:2606. 25293v1 Announce Type: new Abstract: Positional encodings (PEs) are essential for Transformers.

By Yipeng Zhang, Zhongtian Sun, Pietro Li\`o, Kelin Xia

llms benchmarks safety

arXiv Machine Learning

Jun 25

Securing Time Integrity in Energy IoT Against Clock Drift and Y2K38 Failures

arXiv:2601. 23147v2 Announce Type: replace Abstract: The integrity of time in distributed Internet of Things (IoT) devices is crucial for reliable operation in energy cyber-physical systems, such as smart grids and microgrids.

By Saeid Jamshidi, Omar Abdul Wahab, Rolando Herrero, Foutse Khomh

llms rag robotics

arXiv Machine Learning

Jun 25

Bias Fitting to Mitigate Length Bias of Reward Model in RLHF

arXiv:2505. 12843v2 Announce Type: replace Abstract: Reinforcement Learning from Human Feedback (RLHF) relies on reward models to align large language models with human preferences.

By Kangwen Zhao, Jianfeng Cai, Jinhua Zhu, Ruopei Sun, Dongyun Xue, Wengang Zhou, Li Li, Houqiang Li

llms reinforcement-learning safety

arXiv Machine Learning

Jun 25

Neglected Free Lunch from Post-training: Progress Advantage for LLM Agents

arXiv:2606. 26080v1 Announce Type: new Abstract: Process reward models enable fine-grained, step-level evaluation of LLMs, yet building them for agentic settings remains prohibitively difficult: long-horizon interactions, irreversible actions, and stochastic environment feedback make both human annotation and Monte Carlo estimation infeasible at scale.

By Changdae Oh, Wendi Li, Seongheon Park, Samuel Yeh, Tanwi Mallick, Sharon Li

llms agents reinforcement-learning benchmarks

arXiv Machine Learning

Jun 25

Improved Large Language Diffusion Models

arXiv:2606. 25331v1 Announce Type: cross Abstract: Modern large language models are predominantly trained with autoregressive factorization and causal attention.

By Shen Nie, Qiyang Min, Shaoxuan Xu, Zihao Huang, Yuxuan Song, Yong Shan, Yankai Lin, Wayne Xin Zhao, Chongxuan Li, Ji-Rong Wen

llms diffusion fine-tuning benchmarks

arXiv Machine Learning

Jun 25

Lifelong In-Context Learning with Transformers Requires Parametric Forms of Attention

arXiv:2606. 25342v1 Announce Type: new Abstract: Lifelong continual learning remains an obstacle on the path to human-like intelligence.

By Luke McDermott, Robert W. Heath jr., Rahul Parhi

llms agents efficiency

arXiv Machine Learning

Jun 25

Do Thinking Tokens Help with Safety?

arXiv:2606. 25013v1 Announce Type: new Abstract: Today's reasoning models use thinking tokens to attain stronger performance on benchmarks than their instruction-tuned counterparts.

By Narutatsu Ri, Abhishek Panigrahi, Sanjeev Arora

llms benchmarks safety

arXiv Machine Learning

Jun 25

Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Models

arXiv:2606. 25473v1 Announce Type: cross Abstract: Autoregressive video diffusion with causal diffusion transformers has emerged as a major paradigm for real-time streaming video generation and action-conditioned interactive world models.

By Kaiwen Zheng, Guande He, Min Zhao, Jintao Zhang, Huayu Chen, Jianfei Chen, Chen-Hsuan Lin, Ming-Yu Liu, Jun Zhu, Qianli Ma

llms diffusion efficiency benchmarks

arXiv Machine Learning

Jun 25

LLM Evolution as an Industry-Scale Ecosystem: A Lifecycle Perspective on Continual Learning

arXiv:2606. 24901v1 Announce Type: new Abstract: Continual learning capability is critical for Industrial LLMs, as deployed models must be continuously updated to meet evolving requirements and environments, rather than repeatedly retrained from scratch.

By Hao Jiang, Enneng Yang, Guojie Zhu, Yibin Chen, Yunkun Xu, Zifu Kou, Jiayi Li, Chong Chen, Zhao Cao, Li Shen

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 25

RAS: Measuring LLM Safety Through Refusal Alignment

arXiv:2606. 25750v1 Announce Type: cross Abstract: Safety evaluation of large language models (LLMs) is commonly performed by querying models with unsafe or jailbreak prompts and judging whether their outputs violate a safety policy.

By Chang-Chieh Huang, Yan-Lun Chen, Chia-Mu Yu, Wei-Bin Lee

llms safety

Get the daily briefing