MemT-1.0 is a memory-native evolution architecture that embeds memory directly into model computation, using Gated DeltaNet-2 for token-level updates and a Mixture-of-Storage scheme to scale capacity. It is trained in two stages with 34B tokens and released as MemT-1.0-Nano and MemT-1.0-Flash, which outperform parametric memory baselines and compete with retrieval-augmented methods while preserving general capabilities. At a 128K-token history, MemT-1.0-Nano achieves a 1.86× inference speedup and a 66× reduction in persistent storage compared to a full context KV cache.
arXiv:2607. 26760v1 Announce Type: cross Abstract: Recent advances in AI agents have increasingly internalized native capabilities into their underlying foundation models, giving rise to multimodal foundation models and large reasoning models.
arXiv:2610.19210v1 Announce Type: new Abstract: Transformers are efficient to pre-train at scale because attention permits parallel computation across sequence positions, but long-context inference r...
IronLLM-0.6B is a 654‑million‑parameter language model engineered for efficient on‑device inference, featuring a hybrid attention architecture, X‑MTP multi‑token prediction, and a lightweight verification head that yields a 1.48× decoding speedup. Trained on roughly 6.2 trillion tokens with a quality‑oriented pipeline and further refined via Multi‑Domain On‑Policy Distillation, the model adopts an Instruct‑Only design to meet low‑latency requirements. A lighter variant, IronLLM‑0.6B‑Light, replaces RMSNorm with Dynamic Tanh and streamlines costly components to enhance inference and quantization efficiency, offering a strong performance‑efficiency trade‑off for resource‑constrained deployment.
arXiv:2606. 17803v1 Announce Type: new Abstract: Large language models achieve strong reasoning performance by scaling inference-time compute, yet remain fundamentally stateless, discarding the rich, self-produced reasoning traces generated during this process.
arXiv:2608. 12522v1 Announce Type: new Abstract: LLM-based program evolution systems such as FunSearch and AlphaEvolve have shown strong ability to discover novel algorithms, but typically optimize each task in isolation, discarding search experience after completion.
arXiv:2610.23455v1 Announce Type: cross Abstract: LLMs and LVLMs are increasingly deployed across heterogeneous hardware, ranging from data-center GPUs to resource-constrained edge devices. Their fix...
arXiv:2608.23268v1 Announce Type: new Abstract: Frontier multimodal large language models (MLLMs) deliver impressive perception yet still falter on scientific and mathematical reasoning. Parameter-le...
arXiv:2608. 12435v1 Announce Type: new Abstract: Transformers owe much of their strong long-context retrieval capability to a token-level memory that grows with context length.
arXiv:2608. 14621v1 Announce Type: cross Abstract: Long-term memory is increasingly central to LLM agents, yet memory design remains a highly coupled architecture problem: what to encode, how to store it, how to retrieve it, and how to manage it can vary substantially across tasks and backbone models.
RPMem introduces a two‑stage architecture that compiles each session into a model‑independent latent memory and then consolidates it with retained memory via a task‑trained recurrent gate. The consolidated memory is mapped to backbone‑specific low‑rank adaptation (LoRA) parameters, enabling the memory to transfer when the backbone is replaced. Across three long‑term memory benchmarks and five diverse backbones, RPMem achieves broad generalization with near‑constant update cost and memory footprint, outperforming existing parametric and text‑based baselines on the PERMA benchmark.
arXiv:2607. 13591v1 Announce Type: cross Abstract: Large Language Model (LLM) agents increasingly rely on external memory systems to accumulate experience across tasks.
arXiv:2607. 07953v1 Announce Type: cross Abstract: Self-attention lets each token retrieve information from the full context, but its quadratic cost in sequence length limits training and inference at long context.