arXiv Computer Vision By Yanchen Wang, Lenny Aharon, Wangshu Zhu, Kyle Daruwalla, Linghua Zhang, Jiaru Zou, Selmaan Chettih, Helen Hou, Liam Paninski, Matthew R Whiteway

BEAST3D: Animal behavioral analysis and neural encoding from multi-view video via Gaussian splatting

Read the original on arXiv Computer Vision →

BEAST3D is a self‑supervised framework that learns 3D visual representations from unlabeled, calibrated multi‑view video of animals. It uses a vision transformer to predict 3D Gaussian splats, reconstructing held‑out views via differentiable rendering while simultaneously segmenting the animal. The method works with as few as four views, produces viewpoint‑invariant features, and transfers to tasks such as novel view synthesis, multi‑view pose estimation, and neural encoding across four species.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv Computer Vision
Sep 30

Sparse-View Interpretable 3D Animal Behavior Representations for Neural Encoding and Decoding

arXiv:2609.36217v1 Announce Type: new Abstract: A deeper understanding of brain function requires a precise, structured characterization of behavior.Yet, extracting behavioral representations from vi...

By Xinming Dai, Qihang Jin, Tianshu Tan, Baiyuan Chen, Hanrui Lyu, Lenny Aharon, Kyle Daruwalla, Xun Helen Hou, Matthew R. Whiteway, Liam Paninski, Yizi Zhang
arXiv Computer Vision
Sep 30

ORMA: Optimization-based Monocular 4D Reconstruction of Articulated Animals

ORMA is a training‑free framework that reconstructs articulated 4D representations of animals from monocular videos by decoupling pose and shape. It uses predicted pose as a reference for optimization and generative 3D priors to refine shape, aligning the result with the SMAL+ parametric model. The method combines per‑frame pose estimates with globally consistent camera poses, and further refines the reconstruction using self‑supervised DINO correspondences and temporal consistency, achieving improved accuracy on the new PAW4D benchmark and diverse real‑world videos.

By Xuyi Hu, Francesco Palandra, Shangzhe Wu, Daniel Cremers, Riccardo Marin, Silvia Zuffi
arXiv AI
3d ago

Multi-View 3D Reconstruction Using V-JEPA Features

The paper investigates supervised RGB‑only multi‑view 3D reconstruction by integrating video‑pretrained features with camera estimation, scene aggregation, epipolar transport, and residual pointmaps. Experiments on CO3D and indoor datasets compare global versus spatial camera regression and a joint relative‑objective configuration, reporting improvements in trajectory and intrinsics errors but mixed effects on depth accuracy and raw Chamfer distance. The results emphasize that representation adaptation, shape alignment, and raw reconstruction must be evaluated separately before attributing gains to predictive video pretraining.

By Anwar Shamim, Jeffrin Sam
arXiv Computer Vision
Aug 27

PIVOT: A Multi-Trajectory Dataset and Testbed for Pose, Intrinsics, and Novel Viewpoint Evaluation in Real-World 3D Reconstruction

PIVOT is a new multi‑trajectory dataset and evaluation framework that captures real‑world scenes with diverse camera paths, preserving both sensor‑derived measured poses and COLMAP‑optimized poses along with calibrated and optimized intrinsics. It defines three benchmark families—seen vs. unseen trajectory generalization, measured vs. optimized pose sensitivity, and calibrated vs. optimized intrinsics sensitivity—and introduces a directed pose‑space Chamfer distance to assess pose coverage. The first version of PIVOT includes five scenes recorded with a DJI Mini 4 Pro and offers an open processing and Nerfstudio‑based evaluation toolchain, revealing a consistent quality gap between held‑out and unseen trajectories and significant sensitivity to pose source and camera intrinsics.

By Mary Raymond
Hugging Face Trending Papers
6d ago

Multi-View 3D Reconstruction Using V-JEPA Features

The paper investigates supervised RGB‑only multi‑view 3D reconstruction by leveraging video‑pretrained features and a pipeline that includes camera estimation, scene aggregation, epipolar transport, and residual pointmaps. Experiments on CO3D and indoor datasets compare global versus spatial camera regression and a joint relative‑objective configuration, revealing trade‑offs in trajectory, depth, and Chamfer distance metrics. The results emphasize that improvements in reconstruction stem from specific representation adaptations and component interactions rather than solely from predictive video pretraining.