arXiv AI

Event Interaction in Low-Rank Bottlenecks for Temporal Relation Extraction

Hugging Face Trending Papers
Jun 1

WALL-WM: Carving World Action Modeling at the Event Joints

WALL-WM is a World Action Model that shifts video-action learning from chunk-centric optimization to event-grounded Vision-Language-Action pretraining, using semantically coherent action events as the atomic unit of learning. Existing WAMs commonly initialize from multimodal or video foundation models and then optimize fixed-length action chunks conditioned directly on the current observation and instruction.

arXiv Computer Vision
Aug 27

FLEET: Token-Based Feature Extraction for Event Camera-based Reinforcement Learning

FLEET is a token‑based feature extractor that processes event camera data directly, using random Fourier features and cross‑attention to compress variable‑length event streams into fixed‑size latent representations. By decoupling inference cost from sensor resolution, it avoids the high compute and temporal blurring associated with CNN‑based grid aggregation. Experiments on a new high‑throughput benchmark show that FLEET outperforms state‑of‑the‑art methods and remains robust across different observation frequencies.

By Tristan Gottwald, Maximilian Schier, Melanie Schaller, Bodo Rosenhahn
arXiv Computer Vision
Sep 7

Efficient Multi-Timescale Event Representations for Feed-Forward Object Detection

The paper introduces a confidence‑normalized continuous multi‑timescale representation for event cameras, using logarithmic B‑spline temporal encoding and a geometry‑aware local confidence mechanism. When paired with a fixed feed‑forward EventCenterNet detector, this representation outperforms the compact CSTR representation on the PEDRo and Gen1 datasets. Additionally, a recursive exponential‑polynomial approximation is proposed to allow efficient event‑by‑event updates while maintaining detection performance.

By Fredrik Lundell, Per-Erik Forssen, M{\aa}rten Wadenb\"ack, Astrid Lundmark
arXiv Machine Learning
Sep 14

Scaling Online Complex Event Detection with Synthetic Supervision and Mamba-Based Neural Algorithmic Reasoning

The paper presents NAROCE, a Neural Algorithmic Reasoning framework for online complex event detection (CED). It decouples rule learning from sensor semantics by pretraining a Mamba-based rule reasoner on synthetic atomic event traces and then adapting it to raw sensor inputs with limited labeled data. Experiments on a simulator‑generated benchmark show that NAROCE matches or surpasses strong baselines while using far fewer labeled sequences and computational resources.

By Liying Han, Gaofeng Dong, Xiaomin Ouyang, Kang Yang, Lance Kaplan, Federico Cerutti, Mani Srivastava
arXiv AI
Sep 15

EventVL: Understand Event Streams via Multimodal Large Language Model

EventVL introduces the first generative event-based multimodal large language model (MLLM) designed for explicit semantic understanding of event streams. The framework leverages a newly annotated dataset of nearly 1.4 million event–image/video–text pairs and incorporates an Event Spatiotemporal Representation to capture comprehensive event information, along with Dynamic Semantic Alignment to refine sparse semantic spaces. Experiments demonstrate that EventVL outperforms existing MLLM baselines in event captioning and scene description generation tasks, advancing the field of event vision.

By Pengteng Li, Yunfan Lu, Pinghao Song, Wuyang Li, Huizai Yao, Hui Xiong
arXiv AI
Aug 20

Event-Causal RAG: A Retrieval-Augmented Generation Framework for Long Video Reasoning in Complex Scenarios

Event-Causal RAG (EC‑RAG) is a lightweight retrieval‑augmented framework designed for reasoning over ultra‑long and streaming videos. It segments video streams into semantically complete events using a dual visual‑audio sentinel mechanism, representing each event as a State‑Event‑State (SES) structure that captures pre‑event, event, and post‑event states. During question answering, bidirectional graph retrieval accesses relevant predecessor and successor events from a dual vector‑graph memory, and answers are generated using both this structured memory and the corresponding video evidence. The authors also introduce ECV‑1H, an hour‑scale long‑video QA benchmark with over 150 hours of untrimmed video and 1,251 human‑annotated QA pairs, where EC‑RAG achieves significant accuracy gains across multiple video foundation models while maintaining efficient streaming memory usage on a single RTX 5090 GPU.

By Peizheng Yan, Yu Zhao, Liang Xie, Juntong Qi, Mingming Wang, Erwei Yin