← Back to all news
arXiv Computer Vision By Faizan Siddiqui, Nirav Savaliya, Sahith Reddy Chada, Enna Sachdeva

Thinking from Above: Efficient Spatial Reasoning with Bird's-Eye-View Visual Tokens

Read the original on arXiv Computer Vision →

The paper introduces Cambrian-BEV, a parameter‑free geometric tokenization method that projects visual patch features from multiple video frames into a shared bird’s‑eye‑view (BEV) grid, aggregating observations of the same physical location across viewpoints and time. This approach yields a persistent spatial representation while dramatically compressing the visual token set—from 100 K DINOv3 patches across 128 frames to just 729 BEV tokens—reducing LLM prefill time by 2.9× and peak memory by 3.6×. Experiments on VSI‑Bench and ReVSI show that Cambrian‑BEV outperforms frame‑token baselines, achieving significant gains in room‑size and relative‑direction tasks. "whyItMatters":"By providing a stable, compressed spatial context, Cambrian‑BEV enables more efficient and accurate multimodal large language models for embodied agents, improving performance on spatial reasoning benchmarks while reducing computational resources."

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv Computer Vision
Oct 22

Think-Capture: Active 3D Geometry Acquisition for Spatial Reasoning in Vision-Language Models

Think-Capture is a query‑conditioned framework that selectively acquires 3D geometry for task‑critical objects to improve spatial reasoning in Vision‑Language Models. It uses a 3D toolkit to lift zero‑shot masks into a common coordinate frame, a Symbolic 3D State Abstraction to encode object and camera states as structured text, and Hierarchical Group Relative Policy Optimization to train the VLM on when geometry helps or misleads. The resulting Think‑Capture‑4B model achieves significant gains on MindCube‑Tiny, MMSI‑Bench, and ViewSpatial benchmarks while remaining robust to perturbations and fitting on a single 24 GB GPU.

By Junjie Wang, Zhihao Yuan, Yunhan Tang, Sheng Xu, Jiaqi Mao, Xiaoyu Ye, Yueru Luo, Xu Yan, Chun-Mei Feng, Zhen Li
llmsmultimodalsafety
More like this →
Hugging Face Trending Papers
Jul 27

MAViE: A Multi-scale Adaptive Vision Encoder for Fine-grained Visual Perception and Efficient Multimodal Reasoning

Vision-language models commonly project all tokens produced by a pretrained vision encoder into a large language model. However, final-layer features can discard text, local attributes, and spatial relationships, while high-resolution inputs substantially increase context length and inference latency.

llmscomputer-visionefficiencymultimodalbenchmarks
More like this →
arXiv AI
Oct 2

VETO: Video Efficient Token Optimization for Vision Language Models

VETO (Video Efficient Token Optimization for Vision Language Models) is a plug‑in that reduces the quadratic cost of visual tokens in long‑video inference by applying dual‑axis compression: an intra‑frame compressor merges semantically similar tokens within each frame, and an inter‑frame compressor merges temporally redundant frames. By first compressing spatial dimensions, VETO lowers the cost of subsequent global temporal matching, surpassing single‑axis methods and achieving up to 45% faster inference on models such as LLaVA‑OneVision‑7B while maintaining or improving accuracy. The approach is universally applicable across LLaVA‑OneVision, InternVL‑2.5, and LongVA, preserving or enhancing zero‑shot accuracy even under extreme token budgets.

By Gueter Josmy Faure, Hao Ping Wang, Min-Hung Chen, Winston H. Hsu
llmsmultimodal
More like this →