Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

19,764 stories · RSS feed

arXiv Machine Learning

Jul 22

Estimating near-verbatim extraction risk in language models with decoding-constrained beam search

arXiv:2603. 24917v2 Announce Type: replace-cross Abstract: Recent work shows that standard greedy-decoding extraction methods for quantifying memorization in LLMs miss how extraction risk varies across sequences.

By A. Feder Cooper, Mark A. Lemley, Christopher De Sa, Lea Duesterwald, Allison Casasola, Jamie Hayes, Katherine Lee, Daniel E. Ho, Percy Liang

llms safety

arXiv AI

Jul 22

Operational Proto-Introspection in Looped Language Models: Process-Quality Taps, Executable Branching, and the Readout-Control Boundary

arXiv:2607. 18553v1 Announce Type: cross Abstract: Can a language model read the quality of ongoing computation, and can an external intervention turn that readout into better outcomes?

By Jan Kirin

llms fine-tuning

arXiv Machine Learning

Jul 22

Benchmarking Deep Learning Approaches for AEC Engineering Drawing Layout Detection and Information Extraction

arXiv:2607. 18997v1 Announce Type: cross Abstract: Information Extraction (IE) from Architecture, Engineering, and Construction (AEC) drawings remains hindered by manual inefficiency, while Layout Detection, a vital 'middleware' organizing graphical and textual hierarchies, is underexplored.

By Tianyang Huang, Alessio Lombardi, Ahmed Elnagar, Ahmed Zalouk, George Paul, Sepehr Najjarpour, Arvid Sigurdsson, Khalid Ismail, Mohamed Ragab, Edlira Vakaj

llms multimodal benchmarks

arXiv AI

Jul 22

An Analysis of Residual-Stream Geometry Across Transformer Depth

arXiv:2607. 18348v1 Announce Type: cross Abstract: We propose a transition-centred geometric analysis of transformer residual streams.

By Sunit Bhattacharya, Ravi Shankar Kolli

llms

arXiv AI

Jul 22

Dr. Zero: Self-Evolving Search Agents without Training Data

arXiv:2601. 07055v2 Announce Type: replace Abstract: As high-quality data becomes increasingly difficult to obtain, self-evolution without curated training data has emerged as a promising paradigm.

By Zhenrui Yue, Kartikeya Upasani, Xianjun Yang, Suyu Ge, Shaoliang Nie, Yuning Mao, Zhe Liu, Dong Wang

llms agents nlp benchmarks

arXiv Machine Learning

Jul 22

Pain in 3D: Generating Controllable Synthetic Faces for Automated Pain Assessment

arXiv:2509. 16727v5 Announce Type: replace-cross Abstract: Automated pain assessment from facial expressions is crucial for non-communicative patient.

By Xin Lei Lin, Soroush Mehraban, Abhishek Moturu, Babak Taati

llms diffusion computer-vision

arXiv AI

Jul 22

Beyond Single-Dimensional Compression: The Compound Sparsity Frontier of Large Language Models

arXiv:2607. 18280v1 Announce Type: cross Abstract: Large language models (LLMs) are often compressed through static parameter pruning or dynamic token-level computation, yet aggressive sparsification can trigger rapid performance degradation beyond an essential sparsity boundary.

By Chao Han, Haozhe Hu, Xiaoyu Shen

llms nlp efficiency benchmarks

arXiv Machine Learning

Jul 22

HERAKLES: Hierarchical Skill Compilation for Open-ended LLM Agents

arXiv:2508. 14751v2 Announce Type: replace Abstract: We study goal-conditioned reinforcement learning in partially observable environments with sparse rewards and large, structured goal spaces.

By Thomas Carta, Cl\'ement Romac, Loris Gaven, Pierre-Yves Oudeyer, Olivier Sigaud, Sylvain Lamprier

llms agents reinforcement-learning

arXiv AI

Jul 22

OpenRTAG: A Comprehensive Benchmark for Robust Text-Attributed Graph Learning under Data Quality Degradation

arXiv:2607. 19108v1 Announce Type: new Abstract: Text-attributed graphs (TAGs) are an important graph data form that combine relational structure with rich node text.

By Yuze Dai, Zhihan Zhang, Yan Zhao, Ruoyu Wu, Xunkai Li, Zekai Chen, Qiangqiang Dai, Hongchao Qin, Ronghua Li

llms efficiency benchmarks data-engineering

arXiv Machine Learning

Jul 22

ATLAS: A Foundation Neural Sampler for Amorphous Materials

arXiv:2607. 19198v1 Announce Type: cross Abstract: Amorphous materials exhibit exceptional mechanical and functional properties, yet their rugged energy landscapes are notoriously difficult to sample.

By Mouyang Cheng, Denis Blessing, Botao Yu, Gerhard Neumann, Mingda Li, Carles Domingo-Enrich, Yuanqi Du

llms agents diffusion

arXiv AI

Jul 22

Benchmarking Generalization in Financial Statement Fraud Detection: robust evaluation and novel tasks

arXiv:2607. 19259v1 Announce Type: cross Abstract: Financial statement fraud detection (FSFD) is crucial for market integrity but faces challenges from increasingly sophisticated schemes and under-utilized textual data in financial reports.

By Guy Stephane Waffo Dzuyo (Forvis Mazars, LORIA CNRS Universit\'e de Lorraine), Ga\"el Guibon (LORIA CNRS Universit\'e de Lorraine, LIPN CNRS Universit\'e Sorbonne Paris Nord), Christophe Cerisara (LORIA CNRS Universit\'e de Lorraine), Luis Belmar-Letelier (Forvis Mazars)

llms benchmarks

arXiv Machine Learning

Jul 22

On the Diverse Dynamical Behaviors Arising in Deep Linear Transformers

arXiv:2607. 18584v1 Announce Type: new Abstract: We study the inference-time behavior of deep linear encoder-only transformers through the lens of interacting particle systems.

By Sixu Li, Thomas Jacob Maranzatto, Jan Peszek, Trevor Teolis, Semih Akkoc, Konstantin Riedl, Sennur Ulukus, Nicol\'as Garc\'ia Trillos

llms rag

arXiv AI

Jul 22

Context-Masked Truncated Reasoning Audits for Answer-Key Dependence in LLM Tutors

arXiv:2607. 04572v2 Announce Type: replace Abstract: Large language model (LLM) tutors may have access to teacher notes, answer keys, rubrics, or retrieved solutions while producing student-facing explanations.

By Bonan Shen, Dingyan Shang, Youting Wang, Tao Ning, Bowen Liu

llms

arXiv AI

Jul 22

Sequential Learner Modeling Using Multi-Relational Graph Convolutional Networks

arXiv:2607. 19253v1 Announce Type: new Abstract: User modeling is a critical task in a variety of personalized systems.

By Rawaa Alatrash, Mohamed Amine Chatti, Hong Yang, Yumeng Wang

llms rag

arXiv AI

Jul 22

Attributes Should Come from Images, Not Class Names: Distribution-Conditioned Attribute Selection for Vision-Language Models

arXiv:2607. 18695v1 Announce Type: cross Abstract: A popular route to interpretable zero-shot classification asks a large language model (LLM) to describe each class name and prompts CLIP with the resulting descriptors.

By Gautam Rajendrakumar Gare, Jia Shi, Zhiqiu Lin, Deepak Pathak, John Galeotti, Deva Ramanan

llms rag multimodal

arXiv AI

Jul 22

Learning, Reasoning, Refinement: A Framework for Kahneman's Dual-System Intelligence in GUI Agents

arXiv:2506. 17913v2 Announce Type: replace Abstract: Graphical User Interface (GUI) agents have made significant progress in automating digital tasks through the utilization of computer vision and language models.

By Jinjie Wei, Jiyao Liu, Lihao Liu, Ming Hu, Junzhi Ning, Mingcheng Li, Weijie Yin, Junjun He, Xiao Liang, Chao Feng, Dingkang Yang

llms agents computer-vision benchmarks

arXiv Machine Learning

Jul 22

TD-DPO: Difference-Aware Preference Optimization for Mitigating Sycophancy in Clinical Autism Intervention Dialogue

arXiv:2607. 18304v1 Announce Type: new Abstract: The sycophancy of large language models can increase the safety risk in intervention dialogue for autistic children.

By Shuzhong Lai, Junhong Lai, Chenxi Li, Qing Zhou, Haifeng Li, Gang Pan, Lin Yao, Yueming Wang

llms reinforcement-learning fine-tuning safety

arXiv AI

Jul 22

RoboInspector: Unveiling the Unreliability of Policy Code for LLM-enabled Robotic Manipulation

arXiv:2508. 21378v2 Announce Type: replace-cross Abstract: Large language models (LLMs) demonstrate remarkable capabilities in reasoning and code generation, enabling robotic manipulation to be initiated with just a single instruction.

By Chenduo Ying, Linkang Du, Peng Cheng, Yuanchao Shu

llms robotics

arXiv AI

Jul 22

GQLA: Group-Query Latent Attention for Hardware-Adaptive Large Language Model Decoding

arXiv:2605. 15250v3 Announce Type: replace-cross Abstract: Multi-head Latent Attention (MLA), the attention used in DeepSeek-V2/V3, jointly compresses keys and values into a low-rank latent and matches the H100 roofline almost perfectly.

By Fanxu Meng

llms efficiency

arXiv AI

Jul 22

Why Do Vision Language Models Struggle To Recognize Human Emotions?

arXiv:2604. 15280v2 Announce Type: replace-cross Abstract: Understanding emotions is a fundamental ability for intelligent systems to be able to interact with humans.

By Madhav Agarwal, Sotirios A. Tsaftaris, Laura Sevilla-Lara, Steven McDonagh

llms multimodal safety

Get the daily briefing