arXiv Computer Vision

Surgical Kinematics from Monocular Video with Learned Articulated Motion Constraints

arXiv Computer Vision
Sep 28

PICO: Projection-Informed Consistency Optimisation for 6DoF Surgical Tool Pose Estimation

The paper introduces PICO, an end-to-end trainable model for 6DoF surgical tool pose estimation that uses multi-task learning to predict segmentation, depth, and pose parameters. It incorporates two geometry-aware proxy tasks—a projection loss and a point-to-point loss—to enforce consistency in 2D and 3D spaces, improving accuracy and robustness. Evaluated on the SurgRIPE dataset, PICO achieves strong performance, ranking second in rotation accuracy and maintaining competitive translation results, especially under occlusion.

By Lucy Fothergill, Pietro Valdastri, Dominic Jones, Duygu Sarikaya
arXiv Computer Vision
Sep 15

MoCapAnything V2: End-to-End Motion Capture for Arbitrary Skeletons

arXiv:2604.28130v4 Announce Type: replace Abstract: Recent methods for arbitrary-skeleton motion capture from monocular video follow a factorized pipeline, where a Video-to-Pose network predicts join...

By Kehong Gong, Zhengyu Wen, Dao Thien Phong, Mingxi Xu, Weixia He, Qi Wang, Ning Zhang, Zhengyu Li, Guanli Hou, Dongze Lian, Xiaoyu He, Mingyuan Zhang, Hanwang Zhang
arXiv Computer Vision
Aug 25

From Articulated Kinematics to Routed Visual Control for Action-Conditioned Surgical Video Generation

arXiv:2605.08712v2 Announce Type: replace Abstract: Action-conditioned surgical video generation is a critical yet highly challenging problem for robotic surgery. The core difficulty is that low-dime...

By Bohan Li, Shuojue Yang, Baorui Peng, Xianda Guo, Erli Zhang, Youqi Tao, Junfeng Duan, Daguang Xu, Qi Dou, Xin Jin, Wenjun Zeng, Hao Zhao, Yueming Jin
arXiv Computer Vision
Sep 22

SurgMotion: A Video-Native Foundation Model for Universal Understanding of Surgical Videos

SurgMotion is a video-native foundation model that replaces pixel-level reconstruction with latent motion prediction for surgical video analysis. It introduces motion-guided masked prediction, spatiotemporal affinity self-distillation, and spatiotemporal feature diversity regularization to focus on semantically meaningful regions and avoid representation collapse. Trained on SurgMotion-15M, the largest surgical video dataset, it outperforms state-of-the-art methods across 17 benchmarks, improving workflow recognition, action triplet recognition, skill assessment, polyp segmentation, and depth estimation.

By Jinlin Wu, Felix Holm, Chuxi Chen, An Wang, Yaxin Hu, Xiaofan Ye, Zelin Zang, Miao Xu, Lihua Zhou, Huai Liao, Danny T. M. Chan, Ming Feng, Wai S. Poon, Hongliang Ren, Dong Yi, Nassir Navab, Gaofeng Meng, Jiebo Luo, Hongbin Liu, Zhen Lei
arXiv Computer Vision
Nov 6

Kineo: Calibration-Free Metric Motion Capture From Sparse RGB Cameras

Kineo is a fully automatic, calibration‑free pipeline that performs markerless motion capture using unsynchronized, uncalibrated consumer‑grade RGB cameras. It uses 2D keypoints from existing detectors to jointly calibrate cameras (including distortion), reconstruct 3D keypoints, and generate dense scene pointmaps at metric scale. The method introduces confidence‑aware subsampling, graph‑based optimization, and a pairwise reprojection consistency score to achieve robust calibration and accurate metric scale recovery, achieving significant error reductions on EgoHumans and Human3.6M datasets while remaining computationally efficient.

By Charles Javerliat, Pierre Raimbaud, Guillaume Lavou\'e
arXiv Computer Vision
Oct 5

SCOPE-4D: Endoscopic 4D Geometry Foundation Models

SCOPE-4D is an endoscopic 4D geometry foundation model that predicts camera parameters, dense geometry, and 3D tissue trajectories from monocular RGB video in a single forward pass. The authors introduce SCOPE-5K, a curated dataset of about 5,000 real and synthetic gastrointestinal endoscopy and laparoscopy clips, and use it for geometric supervised fine‑tuning. Adding Common–Residual Motion (CRM) constraints and trajectory supervision further improves camera and depth estimation and enables dense 3D tissue tracking, as shown by evaluations on public and new benchmarks and a blinded user study.

By Chaoyi Zhou, Zhongpai Gao, Anwesa Choudhuri, Meng Zheng, Benjamin Planche, Run Wang, Terrence Chen, Siyu Huang, Ziyan Wu