← Back to all news
arXiv Machine Learning By Huajian Ruan, Jinyang Li, Xingyu Guo, Lingxiao Wang

Generative Criticality in Large Language Model Temperature Scaling

Read the original on arXiv Machine Learning →

arXiv:2606. 06238v1 Announce Type: new Abstract: We propose a statistical-field framework for text generated by large language models (LLMs), treating token embeddings as continuous spin variables on a one-dimensional chain.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Aug 31

Deriving Scaling Laws for OpenEuroLLM Models: Learning Rate, Batch Size and Loss

The paper investigates how learning rate and batch size scale when pretraining dense large language models on English‑prevalent corpora, examining both jointly optimal and marginal evolutions across model capacity and data size. It explores the benefits of a Warmup‑Stable‑Decay learning‑rate schedule, assessing whether optimal hyperparameters transfer between stable and decay phases, and evaluates loss scaling forms that capture interactions between model capacity and dataset size. The study provides a baseline scaling procedure and releases the full set of pretraining runs for future OpenEuroLLM development.

By Niccol\`o Ajroldi, Diana Alexandra Onutu, Haider Al-Tahan, J\"org Franke, Sampo Pyysalo, Jenia Jitsev, Aaron Klein
llms
More like this →
arXiv AI
Jul 7

Deriving Neural Scaling Laws from the statistics of natural language

arXiv:2602. 07488v3 Announce Type: replace-cross Abstract: Despite the fact that experimental neural scaling laws have substantially guided empirical progress in large-scale machine learning, no existing theory can quantitatively predict the exponents of these important laws for any modern LLM trained on any natural language dataset.

By Francesco Cagnetta, Allan Ravent\'os, Surya Ganguli, Matthieu Wyart
llmsbenchmarks
More like this →