BEAST3D is a self‑supervised framework that learns 3D visual representations from unlabeled, calibrated multi‑view video of animals. It uses a vision transformer to predict 3D Gaussian splats, reconstructing held‑out views via differentiable rendering while simultaneously segmenting the animal. The method works with as few as four views, produces viewpoint‑invariant features, and transfers to tasks such as novel view synthesis, multi‑view pose estimation, and neural encoding across four species.
By Yanchen Wang, Lenny Aharon, Wangshu Zhu, Kyle Daruwalla, Linghua Zhang, Jiaru Zou, Selmaan Chettih, Helen Hou, Liam Paninski, Matthew R Whiteway
arXiv:2609.36217v1 Announce Type: new
Abstract: A deeper understanding of brain function requires a precise, structured characterization of behavior.Yet, extracting behavioral representations from vi...
By Xinming Dai, Qihang Jin, Tianshu Tan, Baiyuan Chen, Hanrui Lyu, Lenny Aharon, Kyle Daruwalla, Xun Helen Hou, Matthew R. Whiteway, Liam Paninski, Yizi Zhang
ReLift presents a feed‑forward framework that treats dense 3D tracking as a problem of repairing and completing lifted trajectories. By alternating attention along and across trajectories, it corrects depth and correspondence errors, and fills in missing motion, all while training solely on simulated 3D trajectory data. The method improves lifted trajectories from any 2D tracker or geometry model, achieving state‑of‑the‑art accuracy on WorldTrack and running faster than leading 4D trackers.
By Assaf Singer, Maroun Khriesh, Noam Rotstein, Yoni Kasten, Or Litany
The paper introduces a biologically inspired framework that learns object‑centric visual representations from raw videos without human annotations or camera calibration. By using motion boundaries detected via optical flow and clustering to create pseudo‑instance masks, the method supervises a single‑image encoder with pixel‑level pairwise metric learning. Training on 195 million pseudo‑labeled frames and expanding to 421 million frames through Motion‑Verified Self‑Training, the approach yields Swin‑based encoders that outperform or match supervised and self‑supervised baselines on tasks such as monocular depth estimation, 3D object detection, 3D occupancy prediction, and end‑to‑end planning.
By Boshi Li, Xiaohui Wang, Xiaoyang Wu, Zhichao Li, Ya Yang, Naiyan Wang
arXiv:2610.01314v1 Announce Type: new
Abstract: Dynamic scenes may be captured by a moving camera, multiple video streams, or images taken at different times. These observations reveal complementary...
By Ilya Fradlin, Christian Schmidt, Jens Piekenbrinck, Karim Knaebel, Gonzalo Martin Garcia, Bastian Leibe
MoSE3 is a feed‑forward model that predicts dense SE(3) motion—full 6‑DoF rigid transforms—at every pixel from monocular RGB video, providing rotation, translation, and grouping information simultaneously. It achieves this by jointly learning 3D point tracks and rigidity embeddings, then differentiably fitting transforms within soft rigid clusters. The authors also release Art‑Kubric, a large synthetic dataset with dense SE(3) and rigidity labels, and demonstrate state‑of‑the‑art performance on both rigid and articulated benchmarks, with strong generalization to real‑world videos.
By Jiahuan Cheng, Zhiyi Li, Tian Xia, Ruojin Cai, Yilun Du, Qianqian Wang
arXiv:2603.17068v3 Announce Type: replace
Abstract: Structured 3D representations such as keypoints and meshes offer compact, expressive descriptions of deformable objects, jointly capturing geometri...
By Yeheng Zong, Yizhou Chen, Alexander Bowler, Chia-Tung Yang, Ram Vasudevan
TAPVid-MV is a new benchmark for tracking any point in 3D across multiple synchronized camera views. It comprises 284 sequences, 1,142 calibrated camera streams, and 109,769 point tracks, covering indoor and outdoor domains and derived from various modalities such as depth, LiDAR, SLAM, and simulation. The dataset is visually verified, and evaluation shows that current multi‑view trackers do not consistently outperform monocular trackers, highlighting geometry recovery as a key bottleneck.
By Skanda Koppula, Frano Rajic, Abdullah Faiz Ur Rahman, Yi Yang, Ignacio Rocco, Jeet Thakwani, Rishabh Kabra, Andrew Zisserman, Joao Carreira, Siyu Tang, Carl Doersch, Gabriel Brostow
arXiv:2603. 24016v2 Announce Type: replace-cross Abstract: Multi-Object Tracking (MOT) has traditionally focused on a few specific categories, restricting its applicability to real-world scenarios involving diverse objects.
By Zekun Qian, Wei Feng, Ruize Han, Junhui Hou
arXiv:2608.21281v1 Announce Type: new
Abstract: Recent advances in field technology have led to a massive influx of in-the-wild video data for ecological science. The primary bottleneck in leveraging...
By Abigail G. Grassick, Jerome Tze-Hou Hsu, Ethan Lin, Ziang Liu, Max Whitton, Madelyn Hair, Liam Gutierrez, Haozheng Yu, Kristin Branson, Vivek Jayaraman, Michael A. Gil, Andrew M. Hein, Jennifer J. Sun
VastMAT is a large‑scale multi‑animal tracking benchmark featuring 2,947 videos, 337 animal categories, and over 3.6 million bounding boxes with 22,883 identity trajectories. It emphasizes high‑quality, expert‑reviewed annotations and introduces Seen‑category and Unseen‑category evaluation protocols, revealing significant challenges in tracking unseen animals. The authors also propose a lightweight Center‑Distance‑Augmented Association module that boosts HOTA scores for existing MOT methods without extra training.
arXiv:2609.01530v1 Announce Type: new
Abstract: Self-supervised pre-training via cross-view completion learns strong features for 3D vision from co-visible regions of image pairs. However, the refere...
By Thibaut Loiseau, Guillaume Bourmaud, Vincent Lepetit