The State Of LLMs 2025: Progress, Problems, and Predictions

RELISH: LLM REgression with a Latent Iterative State Head

arXiv:2604. 01206v2 Announce Type: replace-cross Abstract: We present RELISH (REgression with a Latent Iterative State Head), a novel, lightweight architecture designed for text regression with large language models.

Fast Inference on Large Language Models: BLOOMZ on Habana Gaudi2 Accelerator

Scaling Point-in-Time Language Models

arXiv:2607. 11889v1 Announce Type: cross Abstract: Large language models trained on unrestricted internet corpora inevitably embed information from the future, introducing lookahead bias that compromises the validity of backtests and causal inference in finance and the social sciences.

A Latent Variable Framework for Scaling Laws in Large Language Models

arXiv:2512. 06553v2 Announce Type: replace-cross Abstract: We propose a statistical framework built on latent variable modeling for scaling laws of large language models (LLMs).

Deriving Scaling Laws for OpenEuroLLM Models: Learning Rate, Batch Size and Loss

The paper investigates how learning rate and batch size scale when pretraining dense large language models on English‑prevalent corpora, examining both jointly optimal and marginal evolutions across model capacity and data size. It explores the benefits of a Warmup‑Stable‑Decay learning‑rate schedule, assessing whether optimal hyperparameters transfer between stable and decay phases, and evaluates loss scaling forms that capture interactions between model capacity and dataset size. The study provides a baseline scaling procedure and releases the full set of pretraining runs for future OpenEuroLLM development.

Cross-LLM Consistency in Inference: Evidence from Shared Interactions

arXiv:2606. 08129v1 Announce Type: new Abstract: Large language models (LLMs) differ in architecture, training data, and optimization procedures, yet they may still develop similar internal inference patterns.

Very Large Language Models and How to Evaluate Them

A Survey on Diffusion Language Models

arXiv:2508. 10875v3 Announce Type: replace-cross Abstract: Diffusion Language Models (DLMs) are rapidly emerging as a powerful and promising alternative to the dominant autoregressive (AR) paradigm.

On the (In)effectiveness of AMR Augmentation for Large Language Models

The paper investigates whether adding Abstract Meaning Representation (AMR) data to large language models (LLMs) improves performance on downstream tasks. By reproducing recent studies and applying a consistent hyperparameter protocol, the authors find that text-only baselines match or surpass AMR-augmented models. A perplexity-based probe shows that AMR does not provide LLMs with additional relational knowledge, suggesting no clear benefit from AMR augmentation.

Accelerating vision-language models with LFM2.5-VL-DSpark

Inverse Depth Scaling From Most Layers Being Similar

arXiv:2602. 05970v2 Announce Type: replace-cross Abstract: Neural scaling laws relate loss to model size in large language models (LLMs), yet depth and width may contribute to performance differently, requiring more detailed studies.

Accelerating Large Language Model Inference with Self-Supervised Early Exits

arXiv:2407. 21082v3 Announce Type: replace-cross Abstract: This paper presents a modular approach to accelerate inference in large language models (LLMs) by adding early exit heads at intermediate transformer layers.