Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

31,384 stories · RSS feed

arXiv Computation and Language
Sep 22

Apollo Restore: A Foundation LLM for Historical Greek Optimized for Fill-in-the-Middle Restoration of Ancient Greek Texts

Apollo Restore is a 24‑billion‑parameter large language model fine‑tuned from Mistral Small to fill in gaps in fragmentary Ancient Greek texts using a fill‑in‑the‑middle objective. It achieves state‑of‑the‑art performance on short lacunae, placing the correct restoration among its top twenty candidates for 80.6% of documentary‑papyrus, 54.6% of literary‑papyrus, and 61.0% of stone‑inscription gaps, outperforming previous models by significant margins. In blind expert evaluations, 20 specialists preferred Apollo Restore over the strongest baseline and judged its performance at least as good as human restorations in 77% of cases, while also improving the published reading of the Vesuvius‑carbonised papyrus P.Herc. 1667.

By Hope McGovern, Anna Dolganov, Samuel Belkadi, Guillaume Kunsch, Dimitris Vlitas, David A. Smith
arXiv Computer Vision
Sep 22

SparkDiffusion: Mitigating the High-Sparsity Trap --- A Unified Framework for up to $265\times$ Single-GPU Acceleration of Visual Generation

arXiv:2609.23153v1 Announce Type: new Abstract: Video diffusion transformers are expensive because attention dominates long spatiotemporal token sequences. We identify the \emph{high-sparsity trap}:...

By Yuxi Liu, Haoyu Li, Zekun Zhang, Tengxu Sun, Yixiang Cai, Jiayong Li, Yifei Xia, Tianle Liu, Baole Ai, Ang Wang, Jiamang Wang, Lin Qu, Kai Zhang, Kun Yuan, Bin Cui
arXiv Machine Learning
Sep 22

FlashBoB: I/O-Efficient Exact Backward-over-Backward for Softmax Attention

FlashBoB introduces an I/O‑efficient algorithm for exact backward‑over‑backward (BoB) in softmax attention, enabling precise second‑order differentiation without large intermediate tensors. By exploiting a hierarchical affine structure, the method confines computation to on‑chip tiles and limits off‑chip memory traffic, achieving θ(N² d²/M) HBM usage. Experiments show FlashBoB scales to sequence lengths of 262K on a single A100 GPU, outperforming prior exact baselines and FlashBack by up to 6.3×.

By Anthony Givans, Michael Crawshaw, Mingrui Liu
arXiv Machine Learning
Sep 22

Spatial-Interactor: Learning Spatial Reasoning through Interaction with the Observable Physical World

arXiv:2609.23038v1 Announce Type: cross Abstract: Spatial reasoning is essential for vision-language models (VLMs) to understand and act in the physical world. Reasoning in dynamic environments requi...

By Kaixiang Yao, Xu Wang, Miao Pan, Hu Xiyue, Weishi Wang, Daniel Dahlmeier, Jintao Chen, Yongliang Shen, Xuhong Zhang, Wenqi Zhang
Simon Willison
Sep 21

Jev introduces a new shape of LLM - System One, aka Decision Models

TypeSafe AI’s new model, Jev, is a ‘System One’ or decision model that takes text or semi‑structured data as input and outputs floating‑point probabilities for yes/no, choice, or score questions, rather than text. It charges only for input tokens ($0.042/million) and offers free output, making it cheaper and faster than typical LLMs. The API lets users build a state object (string, array, or key‑value pairs) and query it with multiple questions, receiving confidence scores and probability distributions for each answer.