arXiv AI
Jul 21

Spatiotemporal Knowledge Graphs as Persistent Scene Memory for Embodied Question Answering

arXiv:2510. 01483v3 Announce Type: replace-cross Abstract: Vision-language models (VLMs) demonstrate strong image-level scene understanding, but reasoning over long egocentric video remains costly: because VLMs maintain no persistent memory or explicit spatial representation, all sampled frames must be re-processed for every new query.

By Mohamad Al Mdfaa, Svetlana Lukina, Timur Akhtyamov, Arthur Nigmatzyanov, Dmitrii Nalberskii, Sergey Zagoruyko, Gonzalo Ferrer
Hugging Face Trending Papers
Jul 22

Memory-Augmented Multimodal Large Language Models for Small Object Understanding in Streaming Aerial Videos

Language-guided aerial perception aims to understand user-specified tiny targets in complex unmanned aerial vehicle (UAV) scenes. In real UAV deployment, the UAV must respond while it flies, so such perception runs in an online streaming manner, where frames arrive sequentially and the model responds to each one without access to future frames.

arXiv AI
2d ago

inr2action: Encoding Robot Episodes as Neural Implicit Representations for Recall and Action

The paper introduces inr2action, a memory framework that encodes entire robot episodes as fixed‑size neural implicit representations. Each episode’s video is fitted to a shared, meta‑learned initialization, producing a weight delta that serves as memory. This delta is projected into token embeddings to condition a language model for question answering and a decoder for command regression, enabling recall without retrieving individual frames.

By Leann Alhashishi, Christos Margadji, Sebastian William Pattinson