arXiv:2609.39785v1 Announce Type: new Abstract: The Segment Anything Model (SAM) relies heavily on massive manual annotations, creating a fundamental bottleneck for model scaling. While unsupervised...
The paper introduces a biologically inspired framework that learns object‑centric visual representations from raw videos without human annotations or camera calibration. By using motion boundaries detected via optical flow and clustering to create pseudo‑instance masks, the method supervises a single‑image encoder with pixel‑level pairwise metric learning. Training on 195 million pseudo‑labeled frames and expanding to 421 million frames through Motion‑Verified Self‑Training, the approach yields Swin‑based encoders that outperform or match supervised and self‑supervised baselines on tasks such as monocular depth estimation, 3D object detection, 3D occupancy prediction, and end‑to‑end planning.
Real-world spatial intelligence requires agents to understand scenes from continuous video streams, where objects move, persist, disappear, and reappear over time. While recent spatial foundation models have enabled generalizable feed-forward 3D reconstruction, most streaming methods remain geometry-centric and lack temporally consistent object-level understanding.
arXiv:2608. 10932v1 Announce Type: cross Abstract: Understanding camera motion is fundamental to video perception, with applications in spatial intelligence and controllable video generation.
DiDA introduces a lightweight video object segmentation framework that leverages Distillation Learning of Deformable Attention. The method uses deformable attention to adapt key and value positions across frames, enabling object representations that are responsive to spatial and temporal changes. Experiments on DAVIS and YouTube‑VOS benchmarks show state‑of‑the‑art performance and efficient memory usage.
SAM‑V is a geometry‑aware extension of the Segment Anything Model (SAM) that integrates 3D priors from a feed‑forward geometry model (VGGT) into 2D segmentation. It uses a prompt‑fusion mechanism to combine sparse SAM prompts with view‑specific camera tokens and local VGGT features, enabling a mask decoder that attends to both dense 2D and 3D cues. The resulting end‑to‑end system produces consistent multi‑view instance segmentation in a single forward pass, achieving significant gains on the IGGT 3D tracking benchmark without offline mask matching or explicit 3D reconstruction.
The paper introduces a promptable localized motion representation that generates persistent embeddings for user-specified regions in a video, without cropping or masking the input. By conditioning motion encoding directly on spatial masks while processing the full video, the method produces temporally consistent, region-addressable embeddings that capture local dynamics while preserving global context. These embeddings enable object-level motion transfer for dynamic scene composition and improve localized action classification in multi-actor videos, outperforming global representations that rely on cropping or post-hoc masking.
arXiv:2609.38714v1 Announce Type: new Abstract: We describe our winning entry to the Waymo Open Dataset 2D Video Panoptic Segmentation Challenge. The task asks for a semantic class at every pixel of...
arXiv:2603. 09420v3 Announce Type: replace-cross Abstract: Motion forecasting enables autonomous vehicles to anticipate scene evolution by predicting the future trajectories of dynamic agents.
CoFiE introduces a two‑stage evidence selection framework for streaming video understanding, separating a coarse, query‑agnostic filtering of visually distinctive frames from a fine, query‑specific refinement during LLM prefill. By filtering out redundant frames before expensive vision encoding, CoFiE reduces end‑to‑end latency while maintaining high accuracy. The method achieves state‑of‑the‑art performance on benchmarks such as StreamingBench and OvO‑Bench, improving accuracy by up to 3.15% and inference speed by up to 2.54× compared to prior approaches.
arXiv:2608.27562v1 Announce Type: new Abstract: Translating continuous, noisy egocentric video streams into discrete, temporally ordered action steps is fraught with visual challenges. Heavy ego-moti...
arXiv:2609.36875v1 Announce Type: new Abstract: Accurate instance segmentation in dynamic scenes is important for downstream applications such as robotics and autonomous driving. Existing Segment Any...