MemT-1.0 is a memory-native evolution architecture that embeds memory directly into model computation, using Gated DeltaNet-2 for token-level updates and a Mixture-of-Storage scheme to scale capacity. It is trained in two stages with 34B tokens and released as MemT-1.0-Nano and MemT-1.0-Flash, which outperform parametric memory baselines and compete with retrieval-augmented methods while preserving general capabilities. At a 128K-token history, MemT-1.0-Nano achieves a 1.86× inference speedup and a 66× reduction in persistent storage compared to a full context KV cache.
Machine-generated by The Flow from the publisher's headline and feed description
— not written or checked by a human. The full article lives at arXiv Machine Learning.
arXiv:2607. 26760v1 Announce Type: cross Abstract: Recent advances in AI agents have increasingly internalized native capabilities into their underlying foundation models, giving rise to multimodal foundation models and large reasoning models.
By Zeyu Zhang, Ziliang Guo, Yihang Sun, Xichong Zhang, Xixuan Hao, Zehao Lin, Yang Zhang, Xiaoyan Zhao, Tong Shen, Bo Tang, Zhi-Qin John Xu, Junchi Yan, Haofen Wang, Xu Chen, Feiyu Xiong, Zhiyu Li, Tat-Seng Chua
arXiv:2610.19210v1 Announce Type: new
Abstract: Transformers are efficient to pre-train at scale because attention permits parallel computation across sequence positions, but long-context inference r...
IronLLM-0.6B is a 654‑million‑parameter language model engineered for efficient on‑device inference, featuring a hybrid attention architecture, X‑MTP multi‑token prediction, and a lightweight verification head that yields a 1.48× decoding speedup. Trained on roughly 6.2 trillion tokens with a quality‑oriented pipeline and further refined via Multi‑Domain On‑Policy Distillation, the model adopts an Instruct‑Only design to meet low‑latency requirements. A lighter variant, IronLLM‑0.6B‑Light, replaces RMSNorm with Dynamic Tanh and streamlines costly components to enhance inference and quantization efficiency, offering a strong performance‑efficiency trade‑off for resource‑constrained deployment.
arXiv:2606. 17803v1 Announce Type: new Abstract: Large language models achieve strong reasoning performance by scaling inference-time compute, yet remain fundamentally stateless, discarding the rich, self-produced reasoning traces generated during this process.
arXiv:2608. 12522v1 Announce Type: new Abstract: LLM-based program evolution systems such as FunSearch and AlphaEvolve have shown strong ability to discover novel algorithms, but typically optimize each task in isolation, discarding search experience after completion.
arXiv:2610.23455v1 Announce Type: cross
Abstract: LLMs and LVLMs are increasingly deployed across heterogeneous hardware, ranging from data-center GPUs to resource-constrained edge devices. Their fix...
By Oscar Kemme, Janek Haberer, Ali Hojjat, Olaf Landsiedel