arXiv:2606. 31495v1 Announce Type: new Abstract: We study a single idea across two settings: that a prediction-error signal, computed by a small predictor over the latent space of a frozen encoder, can serve both as a gate on plasticity and as a substrate for metacognition.
arXiv:2609.07474v2 Announce Type: replace Abstract: Language models compute over tokens: language is their input, their output, and increasingly their internal representation. Whether language should...
arXiv:2606. 27499v1 Announce Type: cross Abstract: Research on agent memory has matured rapidly, but almost entirely on the text side: few existing benchmarks ask, in an interactive environment, when an agent genuinely needs to remember what it saw rather than what it could write down.
ICM-Bench is a new benchmark for evaluating identity-centric reasoning in multimodal agents with long-term memory. It consists of 839 synthetic video clips totaling 141 minutes and 1,217 open-ended questions about six recurring adults in a one-year life album. The benchmark isolates the ability to maintain recurring person identities and reason over their cross-time relations, and compares various baseline systems, showing that while Gemini 3.1 Pro performs well overall, its accuracy drops on questions requiring long-term identity profiles.
The paper introduces GAVAGAI, a method that adds a referential alignment variable to vision‑language models by solving a small assignment problem via entropic optimal transport. By allowing words to be assigned to a special NOTHING column and enforcing a capacity constraint on regions, the approach improves word‑to‑picture accuracy and localization on the SAYCam infant video corpus. Experiments show that the alignment loss boosts performance over a standard next‑token captioning baseline, especially under high non‑reference rates.
arXiv:2605. 28969v2 Announce Type: replace-cross Abstract: If an AI agent makes decisions on a person's behalf, those decisions must align with its user.
The paper introduces Multimodal Filesystem Memory (MFM), a memory architecture for long‑horizon embodied agents that stores episodic, spatial, semantic, procedural, and working memories. MFM uses searchable textual records to link these representations to multimodal episodes, enabling agents to retrieve past experiences via contextual cues and revisit original observations for missing details. Experiments on long‑horizon Minecraft and computer‑use tasks show that MFM improves performance over three base models, achieving higher milestone completion and reward scores compared to baselines.
arXiv:2608. 12627v1 Announce Type: cross Abstract: Long-horizon egocentric memory transforms continuous first-person video and audio into a searchable record of past experiences.
The paper introduces a perception interface that separates vision from language in vision‑language models. A frozen perception stack detects objects, a deterministic semantic serializer converts the perceived state into text, and a standard text‑only large language model (LLM) answers questions. Experiments on a campus‑robot benchmark show that this serialized interface outperforms a zero‑shot VLM of the same language‑model size, especially as the language model shrinks, and that the advantage persists under paraphrase and different supervision regimes.
The paper investigates how new concepts can be integrated into unified multimodal models (UMMs) by separating generation and understanding objectives through a novel visual entity bound to a single task direction. Experiments show that the effectiveness of cross‑task usability depends on where the concept is injected into the shared computation, with a mid‑stack alignment objective achieving high concept acquisition with minimal loss to overall performance. The study highlights that unified weights alone are insufficient; the two directions must share a semantic format at the entry point for efficient concept integration.
EgoMemReason is a new benchmark for week‑long egocentric video understanding that focuses on memory‑driven reasoning rather than simple perception tasks. It tests three memory types—entity, event, and behavior—across 500 questions, each requiring evidence from an average of 5.1 video segments and 25.9 hours of backtracking. Evaluation of 17 models shows that even the best achieves only 39.6% accuracy, highlighting the difficulty of long‑horizon memory in multimodal systems.
arXiv:2607. 11919v1 Announce Type: cross Abstract: Human memory is reconstructive, not a faithful recording.