arXiv:2609.36217v1 Announce Type: new
Abstract: A deeper understanding of brain function requires a precise, structured characterization of behavior.Yet, extracting behavioral representations from vi...
By Xinming Dai, Qihang Jin, Tianshu Tan, Baiyuan Chen, Hanrui Lyu, Lenny Aharon, Kyle Daruwalla, Xun Helen Hou, Matthew R. Whiteway, Liam Paninski, Yizi Zhang
ORMA is a training‑free framework that reconstructs articulated 4D representations of animals from monocular videos by decoupling pose and shape. It uses predicted pose as a reference for optimization and generative 3D priors to refine shape, aligning the result with the SMAL+ parametric model. The method combines per‑frame pose estimates with globally consistent camera poses, and further refines the reconstruction using self‑supervised DINO correspondences and temporal consistency, achieving improved accuracy on the new PAW4D benchmark and diverse real‑world videos.
By Xuyi Hu, Francesco Palandra, Shangzhe Wu, Daniel Cremers, Riccardo Marin, Silvia Zuffi
The paper investigates supervised RGB‑only multi‑view 3D reconstruction by integrating video‑pretrained features with camera estimation, scene aggregation, epipolar transport, and residual pointmaps. Experiments on CO3D and indoor datasets compare global versus spatial camera regression and a joint relative‑objective configuration, reporting improvements in trajectory and intrinsics errors but mixed effects on depth accuracy and raw Chamfer distance. The results emphasize that representation adaptation, shape alignment, and raw reconstruction must be evaluated separately before attributing gains to predictive video pretraining.
By Anwar Shamim, Jeffrin Sam
PIVOT is a new multi‑trajectory dataset and evaluation framework that captures real‑world scenes with diverse camera paths, preserving both sensor‑derived measured poses and COLMAP‑optimized poses along with calibrated and optimized intrinsics. It defines three benchmark families—seen vs. unseen trajectory generalization, measured vs. optimized pose sensitivity, and calibrated vs. optimized intrinsics sensitivity—and introduces a directed pose‑space Chamfer distance to assess pose coverage. The first version of PIVOT includes five scenes recorded with a DJI Mini 4 Pro and offers an open processing and Nerfstudio‑based evaluation toolchain, revealing a consistent quality gap between held‑out and unseen trajectories and significant sensitivity to pose source and camera intrinsics.
By Mary Raymond
The paper investigates supervised RGB‑only multi‑view 3D reconstruction by leveraging video‑pretrained features and a pipeline that includes camera estimation, scene aggregation, epipolar transport, and residual pointmaps. Experiments on CO3D and indoor datasets compare global versus spatial camera regression and a joint relative‑objective configuration, revealing trade‑offs in trajectory, depth, and Chamfer distance metrics. The results emphasize that improvements in reconstruction stem from specific representation adaptations and component interactions rather than solely from predictive video pretraining.
arXiv:2609.38347v1 Announce Type: new
Abstract: Quantifying collective fish behavior requires accurate trajectories, yet multi-view 3D tracking remains challenging due to frequent occlusions, visuall...
By Patt Phurtivilai, Zhiyang Dou, Yifan Wu, Kinfung Chu, Yuan Liu, Lei Yang, Wenping Wang, Taku Komura
The paper introduces a biologically inspired framework that learns object‑centric visual representations from raw videos without human annotations or camera calibration. By using motion boundaries detected via optical flow and clustering to create pseudo‑instance masks, the method supervises a single‑image encoder with pixel‑level pairwise metric learning. Training on 195 million pseudo‑labeled frames and expanding to 421 million frames through Motion‑Verified Self‑Training, the approach yields Swin‑based encoders that outperform or match supervised and self‑supervised baselines on tasks such as monocular depth estimation, 3D object detection, 3D occupancy prediction, and end‑to‑end planning.
By Boshi Li, Xiaohui Wang, Xiaoyang Wu, Zhichao Li, Ya Yang, Naiyan Wang
arXiv:2610.28632v1 Announce Type: cross
Abstract: Video generation models can now depict remarkably realistic motion. Yet it is unclear how to use this motion beyond pixels: to animate a specific 3D...
By Noam Rotstein, Yoad Tewel, Chen Tessler, Haggai Maron, Gal Chechik, Or Litany, Yoni Kasten, Yuval Atzmon
Kirin is a new framework that reconstructs 3D animal motion from in‑the‑wild videos, learns motion priors at scale, and generates realistic motion conditioned on text and image. It introduces AiM3D, the first large‑scale dataset of aligned video‑text‑motion tuples for quadruped animals, and uses an off‑the‑shelf image‑to‑3D model to automatically rig and animate 3D meshes with the generated motion. The framework and dataset provide a foundation for large‑scale, text and image‑conditioned animal motion generation and animation.
By Brian Nlong Zhao, Zhuoyang Pan, James M. Rehg, Jiajun Wu, Shangzhe Wu
SAM 3D Animal is a promptable framework that performs multi‑animal 3D reconstruction from a single image, using the SMAL+ parametric animal model. It jointly reconstructs multiple instances and allows flexible prompts such as keypoints and masks to better handle crowded and occluded scenes. The authors also introduce Herd3D, a new dataset of over 5,000 images that increases species, interaction, and occlusion diversity, and demonstrate state‑of‑the‑art performance on several benchmark datasets.
By Xuyi Hu, Jin Lyu, Jiuming Liu, Yebin Liu, Silvia Zuffi, Liang An, Stefan Goetz
LucidLyra is a framework that reconstructs 3D scenes from video latents in a feed‑forward manner while simultaneously providing open‑vocabulary semantics and instance cues. By distilling knowledge from a camera‑controlled video diffusion model and frozen 2D vision foundation models, it jointly infers 3D Gaussians that encode geometry, appearance, and semantic information without requiring per‑scene optimization or manual 3D annotations. The resulting scenes support novel‑view rendering, open‑vocabulary segmentation, and object grouping, and can be extended to monocular video for dynamic 3D reconstruction.
Free-viewpoint 3D scene media is increasingly important for immersive applications, yet practical capture often suffers from severe view sparsity and motion blur. Although neural rendering has advanced sparse-view synthesis, existing blur-aware methods typically require substantial multi-view redundancy, accurate camera poses, or costly per-scene optimization.