Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

31,046 stories · RSS feed

arXiv AI
Jun 10

FOGO: Forgetting-aware Orthogonalization Optimizer

arXiv:2606. 10406v1 Announce Type: cross Abstract: We argue that forgetting is not confined to continual learning but is a general optimization phenomenon: during standard training, dominant mini-batch gradients suppress rare but useful update directions, causing short-term forgetting at every step.

By Toan Nguyen, Yang Liu, Trung Le, Celso de Melo, Flora D. Salim
arXiv AI
Jun 10

Attention-Discounted Adaptive Sampler for Masked Diffusion Language Models

arXiv:2606. 10829v1 Announce Type: cross Abstract: Masked diffusion language models can reduce inference steps by revealing multiple tokens per denoising iteration, but this parallelism is fragile: positions that are individually confident may be unsafe to commit together when their predictions are coupled.

By Yusuf Sahin, Ahmed Rockey Saikia, Volkan Cevher, Paolo Favaro
arXiv Machine Learning
Jun 10

Influence Dynamics and Stagewise Data Attribution

arXiv:2510. 12071v2 Announce Type: replace Abstract: Current training data attribution (TDA) methods treat the influence one sample has on another as static, but neural networks learn in distinct stages that exhibit changing patterns of influence.

By Jin Hwa Lee, Matthew Smith, Maxwell Adam, Jesse Hoogland
arXiv AI
Jun 10

Provenance-Grounded Gating and Adaptive Recovery in Synthetic Post-Training Data Curation

arXiv:2606. 11127v1 Announce Type: cross Abstract: Synthetic post-training pipelines commonly filter generated samples with reward models or holistic LLM judges, yet two practices remain rarely examined together: whether the filtering signal is grounded in the source evidence that induced each generation, and whether rejected samples can be systematically recovered rather than permanently discarded.

By Soham Bhattacharjee, Karun Sharma, Vinay Kumar Sankarapu, Pratinav Seth