The paper proposes an Infinite-Parameter LLM architecture that generates and adapts its weights from live interaction data using a compact hypernetwork and Bayesian updating, allowing the model to learn from real-time user input rather than relying solely on static pretraining. This approach keeps the stored footprint fixed while effectively enabling an infinite set of weights, potentially improving compute efficiency, freeing context windows, and providing persistent, generalizable knowledge across turns. The authors also outline an evaluation protocol to compare this method against traditional in-context learning and retrieval techniques.
arXiv:2608.23268v1 Announce Type: new Abstract: Frontier multimodal large language models (MLLMs) deliver impressive perception yet still falter on scientific and mathematical reasoning. Parameter-le...
arXiv:2602. 06932v5 Announce Type: replace Abstract: Speculative decoding can significantly accelerate LLM serving, yet most deployments today disentangle speculator training from serving, treating speculator training as a standalone offline modeling problem.
arXiv:2609.00759v1 Announce Type: new Abstract: Large language models (LLMs) increasingly handle in-context learning (ICL) tasks where a long, novel context defines the rules, knowledge, and output s...
arXiv:2606. 07404v1 Announce Type: new Abstract: This paper reports on training a hundred-billion-parameter sparse mixture of experts on a single eight-GPU node, end to end.
arXiv:2606. 25115v1 Announce Type: new Abstract: On-device language-model agents improve by accumulating experience in retrieved memory rather than by updating weights.
APEX-EM is a non‑parametric experience memory that stores full procedural‑episodic traces in a typed Procedural Knowledge Graph and retrieves them via semantic search, structural‑signature matching, and graph traversal. It uses a Plan‑Retrieve‑Generate‑Iterate‑Ingest workflow to produce, quality‑gate, and commit experiences, indexing both successes and failures so the agent learns what to reuse and what to avoid. Evaluations on five benchmarks with a shared GPT‑4o backbone show significant performance gains, such as +7.6 pp on BigCodeBench transfer and +1.4 pp on Lifelong Agent Bench, demonstrating that the memory adds to model capability rather than replacing it.
arXiv:2606. 20002v1 Announce Type: cross Abstract: This work presents a general framework for training large language models (LLMs) to "Connect the Dots" (CoD), a meta-capability required by long-lifecycle agents: as an LLM-based AI agent gets deployed in an environment, it solves a long sequence of tasks while continuously exploring the environment, learning from its own experiences, and iteratively self-updating its context about the environment, thereby achieving progressively better performance on future tasks conditioned on the updated context.
arXiv:2601. 07372v2 Announce Type: replace-cross Abstract: While Mixture-of-Experts (MoE) scales capacity via conditional computation, Transformers lack a native primitive for knowledge lookup, forcing them to inefficiently simulate retrieval through computation.
arXiv:2606. 04536v1 Announce Type: new Abstract: Existing memory-augmented LLM agents store past experience exclusively in prompt space, as textual summaries or retrieved passages, while keeping model parameters frozen throughout a rollout.
arXiv:2607. 08032v1 Announce Type: new Abstract: Large language models, and the agents built on them, spend an ever-growing share of their compute and memory on remembering: caching attention keys and values, carrying long prompts, maintaining recurrent state, and storing what happened in previous turns and sessions.
arXiv:2607. 07847v1 Announce Type: new Abstract: As large language models (LLMs) become increasingly capable, the next question is how can we enable models to continually learn?
The paper introduces LRE (Learned Relevance Eviction), a lightweight, CPU‑only, language‑model‑free scorer that learns which parts of an agent’s interaction history are task‑critical and preserves them verbatim. In experiments, LRE matches or surpasses baseline eviction policies on accuracy‑cost trade‑offs, recovers 93% of full‑history accuracy, reduces worst‑case prompt size by 52%, and outperforms dense and token‑pruning encoders in conversational memory while being 295–1569× smaller. The method also achieves superior budgeted answer quality on LoCoMo reading and can be trained annotation‑free, recovering 95% of supervised scorer performance.