Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

29,741 stories · RSS feed

arXiv Machine Learning

Sep 30

Procedural Core: A Compact Recurrent Initialization for Vision Transformers

arXiv:2609.37631v1 Announce Type: new Abstract: Transformers are typically trained from random initialization, requiring all their capabilities to emerge from large-scale optimization. Recent work sh...

By Zachary Shinnick, Christian Intern\`o, Hemanth Saratchandran, Anton van den Hengel, Damien Teney

llms computer-vision

arXiv Machine Learning

Sep 30

LEMON-ZEST: Evolution-Informed Tokenization for Efficient Protein Language Modeling

arXiv:2609.37675v1 Announce Type: new Abstract: Protein Language Models (PLMs) have made remarkable progress following scaling laws established in natural language processing across sequence- and str...

By Biswajit Banerjee, Claudia Alvarez Carreno, Anton S. Petrov

llms rag nlp benchmarks

arXiv Machine Learning

Sep 30

Behavioral Capacity Certificates for Quantized Language Models

arXiv:2609.37887v1 Announce Type: new Abstract: Activation and key-value cache precision change what a quantized language model computes without altering its stored weights. Direct weight-code bounds...

By Arian Eamaz, Mojtaba Soltanalian

llms efficiency

arXiv Machine Learning

Sep 30

Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation

arXiv:2609.37924v1 Announce Type: new Abstract: Recent work on anchored diffusion language models improves denoising by shaping an intermediate latent space with supervised important-token targets. I...

By Joel Anto Paul, Litu Rout, Aditya Akella, Sanjay Shakkottai

llms diffusion efficiency benchmarks

arXiv Machine Learning

Sep 30

Alpha Diffusion Language Models: Factorization Alone Is Not the Problem

arXiv:2609.38066v1 Announce Type: new Abstract: Discrete diffusion language models can generate multiple tokens in parallel, but reducing the number of denoising steps can lead to inconsistent predic...

By Nikita Gushchin, Dmitry Baranchuk, Alexander Korotin

llms diffusion benchmarks

arXiv Machine Learning

Sep 30

AutoMark: Enabling Autoresearch to Discover Better LLM Watermarks

arXiv:2609.37310v1 Announce Type: cross Abstract: With LLM watermarking being deployed commercially and now required by regulations, improving its reliability and effectiveness has become crucial. Ye...

By Thibaud Gloaguen, Robin Staab, Martin Vechev

llms agents benchmarks

arXiv Machine Learning

Sep 30

Are In-Context Images Worth 10 Dimensions?

arXiv:2609.37659v1 Announce Type: cross Abstract: There has been significant work on understanding the In-Context Learning capabilities of Large Language Models, especially on the induction circuit....

By Adhemar de Senneville, Xavier Bou, J\'er\'emy Anger, Rafael Grompone, Gabriele Facciolo

llms computer-vision

arXiv Machine Learning

Sep 30

Retrieval Capacity of Self-Attention Under Competition

arXiv:2609.37879v1 Announce Type: cross Abstract: How many tokens from its context does a language model actually use, and what determines that number? We study this question through self-attention....

By Timur Mudarisov, Mikhail Burtsev, Radu State

llms

arXiv Machine Learning

Sep 30

Learning What to Remember: Long-horizon Counterfactual Memory Optimization

arXiv:2609.37930v1 Announce Type: cross Abstract: Persistent textual memory allows language models to carry information across long interactions, but learning what to remember is fundamentally a cred...

By Jiaming Tang, Mingyan Liu, Armin Sarabi

llms

arXiv Machine Learning

Sep 30

Embedding Perturbation may Better Reflect Intermediate-Step Uncertainty in LLM Reasoning

arXiv:2602.02427v3 Announce Type: replace Abstract: Large Language Models (LLMs) have achieved significant breakthroughs across various domains, but they can still produce unreliable or misleading ou...

By Qihao Wen, Jiahao Wang, Yang Nan, Pengfei He, Ravi Tandon, Han Xu

llms rag

arXiv Machine Learning

Sep 30

Zero Sum SVD: Balancing Loss Sensitivity for Low Rank LLM Compression

arXiv:2602.02848v2 Announce Type: replace Abstract: Advances in large language models have driven strong performance across many tasks, but their memory and compute costs still hinder deployment. SVD...

By Ali Abbasi, Chayne Thrash, Haoran Qin, Shansita Sharma, Sepehr Seifi, Soheil Kolouri

llms benchmarks

arXiv AI

Sep 30

DScale: Scaling Block-Diffusion Speculative Decoding with Adaptive Verification

arXiv:2609.37532v1 Announce Type: cross Abstract: Growing large language model applications demand efficient inference. At high concurrency, block-diffusion speculative decoding suffers from verifica...

By Rongjian Chen, Minxian Xu, Zhengxin Fang, Kejiang Ye, Chengzhong Xu

llms diffusion efficiency

arXiv AI

Sep 30

TReVS: Integrating Textual Relevance and Visual Saliency for Efficient Vision-Language Model Token Pruning

arXiv:2609.37581v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) excel at visual understanding and reasoning but often incur substantial inference costs due to the large number of visu...

By Jing Wang, Zhiping Wu, Dongdong Ren, Youfang Han, Wei Zhao, Wenbin Li

llms efficiency multimodal benchmarks

arXiv AI

Sep 30

JudgeProfile: Understanding and Steering Subjectivity in LLM Judges

JudgeProfile is a framework that analyzes the subjectivity of large language model (LLM) judges by separating evaluation into perception—how judges compare responses on attributes such as clarity, correctness, and detail—and prioritization—how much each attribute influences the final decision. Using the curated SubjectiveSet dataset of 50,013 response pairs evaluated by 21 judges across 87 attributes, the study finds that judges often agree on attribute judgments even when their overall choices differ. By estimating and adjusting attribute weights, the authors improve agreement with reference labels from 66.48% to 71.97%, outperforming fine‑tuning and rubric prompting.

By Qi Cao, Kangning Liu, Xuan Kan, Shunwen Tan, Yang Pei, Dake Chen, Yatai Ji, Zixuan Ye, Yuanpeng Tu, Daniel Li, Junbiao Tang, Pengtao Xie, Zihao He

llms fine-tuning

arXiv AI

Sep 30

Infrared Subtraction with Artificial Intelligence

The article introduces an AI‑developed local infrared subtraction method that separates integrable radiation from a finite Born‑contact term using EFT singular distributions in observables like N‑jettiness. Two implementations are presented: one employing a neural‑network phase‑space projection fitted to EFT cumulants, and another using an analytic construction that keeps Born momenta fixed while integrating over radiation. The method is demonstrated by reconstructing NLO corrections for massless 3‑ and 4‑jet production in electron‑positron annihilation and extending to NNLO dijet production, with results agreeing with EERAD3 and showing feasibility on modest hardware.

By Wenjie He, Xiaohui Liu, Yandong Liu, Zhan Wang

llms