arXiv Computer Vision

ORMA: Optimization-based Monocular 4D Reconstruction of Articulated Animals

ORMA is a training‑free framework that reconstructs articulated 4D representations of animals from monocular videos by decoupling pose and shape. It uses predicted pose as a reference for optimization and generative 3D priors to refine shape, aligning the result with the SMAL+ parametric model. The method combines per‑frame pose estimates with globally consistent camera poses, and further refines the reconstruction using self‑supervised DINO correspondences and temporal consistency, achieving improved accuracy on the new PAW4D benchmark and diverse real‑world videos.

arXiv Computer Vision
Oct 28

PRIMA: Boosting Animal Mesh Recovery with Taxonomic Priors and Adaptation-Driven Data Bootstrapping

PRIMA is a framework for recovering articulated 3D animal meshes from single images. It introduces taxonomic shape priors derived from BioCLIP image features to embed species-level knowledge, and an adaptation-driven bootstrapping method that converts abundant 2D annotations into reliable pseudo-3D supervision, producing the Quadruped3D dataset with about 9K pseudo-3D annotations. These components address the long-tailed distribution of species and limited 3D supervision, achieving state‑of‑the‑art performance on several animal mesh reconstruction benchmarks, especially for rare species and challenging poses.

By Xiaohang Yu, Ti Wang, Mackenzie Weygandt Mathis
arXiv AI
Oct 9

SAM 3D Animal: Promptable Animal 3D Reconstruction from Images in the Wild

SAM 3D Animal is a promptable framework that performs multi‑animal 3D reconstruction from a single image, using the SMAL+ parametric animal model. It jointly reconstructs multiple instances and allows flexible prompts such as keypoints and masks to better handle crowded and occluded scenes. The authors also introduce Herd3D, a new dataset of over 5,000 images that increases species, interaction, and occlusion diversity, and demonstrate state‑of‑the‑art performance on several benchmark datasets.

By Xuyi Hu, Jin Lyu, Jiuming Liu, Yebin Liu, Silvia Zuffi, Liang An, Stefan Goetz
arXiv Computer Vision
Nov 6

BEAST3D: Animal behavioral analysis and neural encoding from multi-view video via Gaussian splatting

BEAST3D is a self‑supervised framework that learns 3D visual representations from unlabeled, calibrated multi‑view video of animals. It uses a vision transformer to predict 3D Gaussian splats, reconstructing held‑out views via differentiable rendering while simultaneously segmenting the animal. The method works with as few as four views, produces viewpoint‑invariant features, and transfers to tasks such as novel view synthesis, multi‑view pose estimation, and neural encoding across four species.

By Yanchen Wang, Lenny Aharon, Wangshu Zhu, Kyle Daruwalla, Linghua Zhang, Jiaru Zou, Selmaan Chettih, Helen Hou, Liam Paninski, Matthew R Whiteway
arXiv Computer Vision
Sep 3

Kirin: Animal Motion Generation from In-the-Wild Video

Kirin is a new framework that reconstructs 3D animal motion from in‑the‑wild videos, learns motion priors at scale, and generates realistic motion conditioned on text and image. It introduces AiM3D, the first large‑scale dataset of aligned video‑text‑motion tuples for quadruped animals, and uses an off‑the‑shelf image‑to‑3D model to automatically rig and animate 3D meshes with the generated motion. The framework and dataset provide a foundation for large‑scale, text and image‑conditioned animal motion generation and animation.

By Brian Nlong Zhao, Zhuoyang Pan, James M. Rehg, Jiajun Wu, Shangzhe Wu
arXiv Computer Vision
Sep 30

Sparse-View Interpretable 3D Animal Behavior Representations for Neural Encoding and Decoding

arXiv:2609.36217v1 Announce Type: new Abstract: A deeper understanding of brain function requires a precise, structured characterization of behavior.Yet, extracting behavioral representations from vi...

By Xinming Dai, Qihang Jin, Tianshu Tan, Baiyuan Chen, Hanrui Lyu, Lenny Aharon, Kyle Daruwalla, Xun Helen Hou, Matthew R. Whiteway, Liam Paninski, Yizi Zhang
arXiv Computer Vision
Sep 11

Artic-O: End-to-End Articulated Object Reconstruction via Latent Geometry Learning

Artic-O is an end‑to‑end, feed‑forward framework that reconstructs articulated objects from sparse images by learning latent geometry. It maps multi‑state observations into a pretrained latent geometry space, uses a frozen flow‑matching decoder for complete‑shape priors, and fuses visual tokens with geometry latents in an image‑grounded part‑reasoning module to segment active parts and predict articulation. Trained with a geometry‑to‑articulation curriculum and a decoupled two‑pass strategy, Artic‑O achieves high reconstruction quality and articulation accuracy while drastically reducing inference time from 9 minutes to about 0.3 seconds per object.

By Xuyang Wang, Zhenyu Li, Jian Ding, Habib Slim, Peter Wonka, Hongdong Li, Mohamed Elhoseiny
arXiv Computer Vision
Sep 14

UniMo: Unifying Human and Animal Motion Generation

UniMo introduces a unified point‑cloud based framework for generating 3D motion that works for both humans and animals, overcoming challenges posed by diverse skeletal topologies and limited animal datasets. It converts parametric skeletons into unparametric representations and uses dynamic sampling to focus on active joints. The authors also release UniML3D, a large motion‑language dataset with 145,907 sequences and 433,388 captions, and demonstrate state‑of‑the‑art performance on multiple benchmarks.

By Zeyu Zhang, Zhiyuan Zhang, Siheng Wang, Yiran Wang, Danning Li, Ian Reid, Richard Hartley
arXiv Computer Vision
Oct 21

CARI4D: Category Agnostic 4D Reconstruction of Human-Object Interaction

CARI4D is a category‑agnostic method that reconstructs spatially and temporally consistent 4D human‑object interactions from monocular RGB videos at metric scale. It introduces a pose hypothesis selection algorithm that integrates predictions from foundation models, refines them via a learned render‑and‑compare paradigm, and reasons about contacts to satisfy physical constraints. Experiments demonstrate a 38% improvement on in‑distribution data and 36% on unseen data compared to prior work, and the model generalizes to unseen categories, enabling zero‑shot application to internet videos.

By Xianghui Xie, Bowen Wen, Yan Chang, Hesam Rabeti, Jiefeng Li, Ye Yuan, Gerard Pons-Moll, Stan Birchfield
arXiv Computer Vision
Oct 9

GenIA: Generative Reconstruction with Test-Time Input Alignment

GenIA is a framework that aligns generative 3D foundation models with test‑time observations, improving pose estimation and reconstruction from monocular, multi‑view, and dynamic inputs. It derives translation and scale from geometry, retains learned rotation priors, and aligns appearance using visibility‑biased attention, cross‑observation fusion, and differentiable rendering guidance during denoising. An optional post‑denoising refinement further adapts appearance latents and object placement, and the method also supports externally supplied geometry for dynamic objects, achieving better results than recent optimization‑based and image‑to‑3D methods on synthetic and real benchmarks.

By Stefano Esposito, Naama Pearl, Polina Karpikova, Samuel Rota Bul\`o, Lorenzo Porzi, Peter Kontschieder, Andreas Geiger, Jonathon Luiten
arXiv Computer Vision
Sep 15

MoCapAnything V2: End-to-End Motion Capture for Arbitrary Skeletons

arXiv:2604.28130v4 Announce Type: replace Abstract: Recent methods for arbitrary-skeleton motion capture from monocular video follow a factorized pipeline, where a Video-to-Pose network predicts join...

By Kehong Gong, Zhengyu Wen, Dao Thien Phong, Mingxi Xu, Weixia He, Qi Wang, Ning Zhang, Zhengyu Li, Guanli Hou, Dongze Lian, Xiaoyu He, Mingyuan Zhang, Hanwang Zhang