arXiv Computer Vision

Bridging the Markerless Tracking Validation Gap: Intraoperative X-ray Workflow Integration

The paper presents a clinically viable pipeline that uses routine intraoperative C‑arm fluoroscopy to derive per‑vertebra 6‑DoF poses, which are co‑registered to an RGB‑D camera via a multimodal fiducial array. This approach provides independent subsurface ground truth for validating markerless spinal tracking methods without adding radiation or altering standard surgical workflows. Ex vivo tests show an end‑to‑end accuracy of about 2 mm and 3.3°, and the framework is applied to evaluate two markerless registration strategies on real surgical data.

arXiv Computer Vision
Sep 28

PICO: Projection-Informed Consistency Optimisation for 6DoF Surgical Tool Pose Estimation

The paper introduces PICO, an end-to-end trainable model for 6DoF surgical tool pose estimation that uses multi-task learning to predict segmentation, depth, and pose parameters. It incorporates two geometry-aware proxy tasks—a projection loss and a point-to-point loss—to enforce consistency in 2D and 3D spaces, improving accuracy and robustness. Evaluated on the SurgRIPE dataset, PICO achieves strong performance, ranking second in rotation accuracy and maintaining competitive translation results, especially under occlusion.

By Lucy Fothergill, Pietro Valdastri, Dominic Jones, Duygu Sarikaya
Hugging Face Trending Papers
Jul 15

Marker-free deformable registration and fusion for augmented reality-guided positive margin localization during tumor resection surgery

Positive margins in head and neck oncologic surgery require mapping specimen-side pathology findings to the patient resection bed. This is challenging because pathologists identify the positive margin on slices of the resected, deformed specimen, while surgeons must relocate the corresponding site on the resection bed using only verbal descriptions and no visual guidance.

arXiv Computer Vision
Sep 17

Video-Based Markerless Motion Capture for Clinical and Rehabilitation Biomechanics: A PRISMA-ScR Scoping Review of Validated Architectures, Clinical Readiness, and Emerging Methods

This scoping review examined 117 studies on video-based markerless motion capture, most published from 2024 onward and focused on healthy adults walking in laboratories. The studies identified five main pipeline architectures, but most reported only raw joint angles without biomechanical refinement, achieving sagittal lower‑limb agreement of about 5–6°, which falls short of clinical acceptability. Validation of out‑of‑plane kinematics, kinetics, and performance in older or pathological populations was rare, and emerging computer‑vision techniques such as foundation‑model mesh recovery and differentiable inverse kinematics were largely absent from validated work.

By Florian Delaplace (LAMHESS, CHU), Elodie Piche (LAMHESS), Fr\'ed\'eric Chorin (IUF, LAMHESS), Raphael Zory (IUF, LAMHESS)
arXiv Computer Vision
Sep 3

MV-dVRK: A Multi-Viewpoint Benchmark for Spatial Surgical Perception

MV-dVRK is the first ex‑vivo surgical dataset that provides multiple exposure‑synchronized stereo viewpoints, accurate surface geometry, and ground‑truth camera poses for endoscopic images. The benchmark’s static subset offers dense SfM reference geometry validated against an industrial 3D scanner, while the dynamic sequences cover ten surgical tasks with increasing kinematic complexity and tissue deformation. Using MV‑dVRK, the authors systematically compare zero‑shot monocular, stereo, multi‑stereo, and multi‑view 3D reconstruction methods, finding that multi‑stereo reconstruction with two endoscopes yields the highest coverage, and that optimization‑based multi‑view methods outperform feed‑forward foundation models when a third viewpoint is added.

By Guido Caccianiga, Sergey Prokudin, Yutong Chen, Bernard Javot, Rachael L'Orsa, Omer Burak Alada\u{g}, Yarden Sharon, Jens Rolinger, Ivan Capobianco, Anton Deguet, Siyu Tang, Katherine J. Kuchenbecker
arXiv Computer Vision
Aug 26

C3VDReg: A Benchmark for Local-to-Local Colonoscopic Registration toward Anatomical Localization

C3VDReg is a benchmark for local-to-local colonoscopic registration that uses the Colonoscopy 3D Video Dataset (C3VD) to generate 10,015 partial-to-partial point cloud pairs, with 2,088 held‑out test pairs. Each pair consists of a source point cloud from depth reprojection and a target point cloud from CT mesh raycasting, evaluated under a standardized protocol of 8,192 points per cloud and fixed pose conventions. Experiments show that high geometric overlap does not guarantee reliable pose recovery, revealing translation ambiguity along repetitive tubular anatomy as a key failure mode.

By Linzhe Jiang, Jiayuan Huang, Sophia Bano, Matthew J. Clarkson, Zhehua Mao, Mobarak I. Hoque