Surgical teams could benefit from assistants that watch the video for requested events, such as "tell me when the clip applier appears during Calot triangle dissection", and answer questions about wha...
PROVIA is a system for online mistake detection in egocentric videos that tracks procedure state by maintaining a factual state and a learned summary of performed steps, including mistakes. It uses Bayesian state merging to create an automaton from correct demonstrations and applies a sequential test to convert per‑frame mistake probabilities into alarms. Evaluated on datasets such as CaptainCook4D, IndustReal, HoloAssist, and IMPACT‑ego, PROVIA outperforms baseline methods while operating at 58–70 fps.
By Di Wen, Kailun Yang, Jimmy Weissert, Luc Maria Scherrer, Cedric Z\"ollner, Ruiping Liu, Yufan Chen, Jiale Wei, Junwei Zheng, Kunyu Peng
TRACE (Temporal Audit and Condition-aware Evaluation) is a new benchmark and evaluation framework for streaming video understanding that explicitly records when evidence becomes valid, how visual history is maintained, and how responses are triggered. It combines temporally audited visual tasks, evidence timing, instruction-dependent trigger annotations, a unified causal Core–Adapter protocol, and multidimensional reporting of answer quality, timeliness, response-selection behavior, workload, completion, and reliability. Using 1,240 records from 517 videos, TRACE evaluated eight publicly available models in eight configurations, revealing that similar QA accuracy can hide significant differences in completion, answer validity, generation workload, and proactive performance metrics such as response delay, false alarms, and missed target windows.
By Yibo Ma, Qianqian Zhang, Peng Liu, Tiancheng Zhao
arXiv:2603.29962v4 Announce Type: replace
Abstract: Surgical procedures are inherently complex and risky, requiring extensive expertise and constant focus to navigate evolving intraoperative scenes....
By Shi Li, Vinkle Srivastav, Nicolas Chanel, Saurav Sharma, Nabani Banik, Lorenzo Arboit, Kun Yuan, Pietro Mascagni, Nicolas Padoy
arXiv:2610.34926v1 Announce Type: new
Abstract: An action seen once and the same action seen three times receive the same action-recognition label, so that supervision does not require a model to cou...
By Yihao Hu, Pan Wang, Yitong Lin, Zhihao Wen, Deyuan Liu, Hang Wang, Mingda Wan, Yiwen Zhao, Yutong Li, Yuchen Mao, Tao Lin
arXiv:2608. 14015v1 Announce Type: cross Abstract: Understanding tens-of-minutes surgical videos requires long-horizon temporal reasoning, answering what happens before, after, or across stages of a procedure by grounding the question in visual evidence spread across time.
By Yingying Fan, Penghui Du, Leyan Zhu, Runze He, Zimeng Wu, Yuxuan Zhang, Liang Chen, Jiahao Xie, Jiangtang Wang, Shuai Shao, Anchao Yang, Yutong Bai, Yan Wang