arXiv Machine Learning By Jinliang Gao, Ning Yang, Hai Wang, Baili Xiao, Pin Lyu

SDO: Structure-Aware Data Organization for Efficient LLM Post-Training

Read the original on arXiv Machine Learning →

arXiv:2607. 27273v1 Announce Type: new Abstract: Post-training of large language models is expensive, and existing efficiency improvements mainly focus on selecting informative samples or designing training schedules.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
Aug 4

Structured Recurrent Mixers for Massively Parallelized Sequence Generation

arXiv:2605. 08696v4 Announce Type: replace-cross Abstract: Over the last two decades, language modeling has experienced a shift from the use of predominantly recurrent architectures that process tokens sequentially during training and inference to non-recurrent models that process sequence elements in parallel during training, which results in greater training efficiency and stability at the expense of lower inference throughput.

By Benjamin L. Badger