arXiv AI
Jun 19
DF3DV-1K: A Large-Scale Dataset and Benchmark for Distractor-Free Novel View Synthesis
arXiv:2604. 13416v2 Announce Type: replace-cross Abstract: Advances in radiance fields have enabled photorealistic novel view synthesis.
Attention-DP3: Spatially Object-aware 3D Diffusion Policy via Geometry-aligned Attentional Conditioning
Related stories
arXiv Computer Vision
5d ago
T3R: Training-Free Two-Stage Token Refinement Towards Efficient and Robust VLA Models
T3R is a training‑free, two‑stage token refinement framework designed to improve the efficiency and robustness of Vision‑Language‑Action (VLA) models. The first stage uses segmentation‑guided pruning to remove irrelevant visual patches, while the second stage applies saliency‑guided attention biasing to focus on action‑critical regions and language cues. Experiments on LIBERO‑Spatial, LIBERO, SIMPLER, and RoboTwin show that T3R can eliminate 85% of visual tokens with minimal loss in success rate and even improve performance under distribution shifts, achieving up to a 3.3× speedup in large‑scale settings.
arXiv AI
Oct 27
Generalized Decoupled Learning for Enhancing Open-Vocabulary Dense Perception
arXiv:2508.11256v2 Announce Type: replace-cross Abstract: Dense visual perception tasks have been constrained by their reliance on predefined categories, limiting their applicability in real-world sc...
arXiv Machine Learning
Sep 10
CoVeR: Coverage-Based Token Pruning for Multi-View 3D Reasoning in VLMs
arXiv:2609.08345v1 Announce Type: cross Abstract: Representing a 3D scene as multi-view images allows 2D VLMs to reason in 3D by reusing priors from pre-training, sidestepping the scarcity of annotat...
arXiv AI
Jul 29
SAM3D-Guided Object-Centric Representation Alignment for Vision-Language-Action Models
arXiv:2607. 25912v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have shown strong potential for general robot manipulation, but most existing models rely on 2D visual-language backbones and lack fine-grained 3D understanding of target objects, especially under occlusion, pose variation, scale changes, and precise spatial interaction.
arXiv AI
Sep 21
FOCAL-VLA: Subtask-Guided Geometry Distillation and Implicit World Modeling for Vision-Language-Action Models
FOCAL‑VLA is a framework that improves vision‑language‑action models by combining subtask‑guided geometry distillation with implicit world modeling. It transfers geometric knowledge from VGGT to focus on subtask‑relevant image regions and uses Track4World features to capture future 3D evolution, guiding action generation without running these models at inference time. Experiments demonstrate that FOCAL‑VLA outperforms baselines on both simulation benchmarks and real‑world manipulation tasks.
arXiv Computer Vision
Sep 18
GAPrompt++: Multi-Granular Geometry-Aware Point Cloud Prompt for 3D Vision Model
GAPrompt++ is a multi-granular geometry-aware prompting method designed to adapt pre-trained 3D vision models to downstream tasks efficiently. It introduces a Point Shift Prompter for multi-scale geometric feature extraction, a Keypoint Prompter for local geometric saliency, and a Prompt Propagation mechanism to embed these cues throughout the model hierarchy. Experiments demonstrate that GAPrompt++ outperforms other prompting-based PEFT methods and even surpasses full fine-tuning while using less than 2% trainable parameters, and the authors provide two new challenging benchmarks for future research.
arXiv AI
Oct 5
OmniAct3D: Leveraging Foundation Geometry and Evidence-Grounded Reasoning for Panoramic 3D Detection
OmniAct3D is a framework that adapts Vision Foundation Model (VFM) detectors trained on perspective images to equirectangular projection (ERP) panoramic images for 3D detection. It introduces the ERP-Ray Geometry Adapter (ERGA-Ray) to model spherical rays, the Visual-Action Reasoning Chain (VARC) to ground hypotheses in panoramic evidence, and the Appearance-Guided Heading Expert (AGHE) to recover high‑resolution heading cues. The method achieves a 2.96 NDS improvement on Spheriverse and a 24.87 mAP gain on PanoMMOcc, while preserving most of the baseline’s performance across configurations.
arXiv AI
Oct 23
AFFORD2ACT: How Far Can Sparse Keypoints Go for Robot Manipulation?
AFFORD2ACT investigates whether robot manipulation policies can rely on a very compact visual representation instead of dense images. Using an affordance model to select up to 15 object keypoints and 4 end‑effector keypoints, the policy receives only 19 2D points (38 coordinates). In six real‑robot tasks, this sparse keypoint input outperforms RGB, RGB‑D, and dense‑affordance baselines, maintains performance with fewer demonstrations, and achieves 82% success on held‑out data, though it depends on accurate affordance localization and keypoint tracking.
arXiv Computer Vision
Aug 31
Video Generative Models as Geometry Learner
The paper introduces GeoNeXt, a framework that repurposes pretrained video generative models for geometry estimation by framing it as a next‑frame prediction task. Unlike prior methods that either train separate depth/normal models or fine‑tune image diffusion backbones, GeoNeXt jointly models images and geometric targets, leveraging the structured knowledge of video models for more data‑efficient learning. Experiments show zero‑shot monocular depth and surface normal estimation that outperforms existing generative approaches and rivals discriminative state‑of‑the‑art methods while using far less training data.
arXiv Computer Vision
Oct 22
Think-Capture: Active 3D Geometry Acquisition for Spatial Reasoning in Vision-Language Models
Think-Capture is a query‑conditioned framework that selectively acquires 3D geometry for task‑critical objects to improve spatial reasoning in Vision‑Language Models. It uses a 3D toolkit to lift zero‑shot masks into a common coordinate frame, a Symbolic 3D State Abstraction to encode object and camera states as structured text, and Hierarchical Group Relative Policy Optimization to train the VLM on when geometry helps or misleads. The resulting Think‑Capture‑4B model achieves significant gains on MindCube‑Tiny, MMSI‑Bench, and ViewSpatial benchmarks while remaining robust to perturbations and fitting on a single 24 GB GPU.
arXiv Machine Learning
Sep 23
SAM-V: Geometry-Aware Segment Anything for Multi-View Instance Segmentation
SAM‑V is a geometry‑aware extension of the Segment Anything Model (SAM) that integrates 3D priors from a feed‑forward geometry model (VGGT) into 2D segmentation. It uses a prompt‑fusion mechanism to combine sparse SAM prompts with view‑specific camera tokens and local VGGT features, enabling a mask decoder that attends to both dense 2D and 3D cues. The resulting end‑to‑end system produces consistent multi‑view instance segmentation in a single forward pass, achieving significant gains on the IGGT 3D tracking benchmark without offline mask matching or explicit 3D reconstruction.