arXiv AI October 16, 2026 By Zane Durante, Paul Tang, Keshigeyan Chandrasegaran, Amit Kaushal, Shrinidhi Kowshika Lakshmikanth, Feolu Kolawole, Yu Jun Zheng, Yifan Geng, Shobhit Agarwal, Kevin Schulman, Juan Carlos Niebles, Ehsan Adeli, Arnold Milstein, Li Fei-Fei

TRACER: A Multimodal Harness for Patient Care in Hospitals

Read the original on arXiv AI →

TRACER is a training‑free multimodal harness that builds minute‑level action records from continuous video and merges them with clinical data into hourly reports, enabling an LLM agent to query and inspect both modalities. In evaluation, TRACER achieved up to 84.8% accuracy on action detection and 61.0% accuracy on clinician‑informed QA, outperforming the strongest baseline. Across 280 patients and ~60,000 camera‑hours, TRACER detected significantly less purposeful activity in patients with a positive delirium assessment compared to those without (p < 0.0001).

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jul 3

MedStreamBench: A Time-Aware Benchmark for Streaming and Proactive Medical Video Understanding

arXiv:2607. 01751v1 Announce Type: cross Abstract: Existing medical video benchmarks primarily evaluate whether a model produces the correct answer, but rarely assess whether it answers at the right time.

By Yuan Wang, Shujian Gao, Songtao Jiang, Zhengyu Hu, Zuozhu Liu

llms multimodal benchmarks

arXiv Computation and Language

Aug 25

Checkup2Action: A Multimodal Clinical Check-up Report Dataset for Patient-Oriented Action Card Generation

arXiv:2605.11533v4 Announce Type: replace Abstract: Routine clinical check-up reports combine laboratory measurements, physiological assessments, imaging findings and visually structured information,...

By Sike Xiang, Shuang Chen, Kevin Qinghong Lin, Jialin Yu, Yijia Sun, Philip Torr, Amir Atapour-Abarghouei

llms multimodal benchmarks safety

arXiv AI

Aug 28

Structured Evidence Routing for Incident Risk Prediction from Multimodal Longitudinal EHRs

The paper introduces structured evidence routing for incident risk prediction using multimodal longitudinal electronic health records (EHRs). It proposes a router‑predictor‑reviewer workflow that condenses full patient records into compact summaries and targeted evidence slices, enabling a predictor to generate evidence‑linked risk assessments that a reviewer can critique. Experiments on five one‑year incident diagnosis tasks show that the method matches the AUROC of established supervised EHRSHOT baselines and remains competitive on AUPRC, while providing a patient‑specific evidence trail.

By Animesh Agarwal, Meysam Ghaffari, Nina Fatehi, Carlos Morato

multimodal

arXiv Machine Learning

Aug 4

EHR2Path: Comprehensive Pathway-Level Modeling of Longitudinal Patient Trajectories from Multimodal Electronic Health Records

arXiv:2506. 04831v3 Announce Type: replace Abstract: Forecasting how a patient's condition is likely to evolve, including possible deterioration, recovery, treatment needs, and care transitions, could support more proactive and personalized care, but requires modeling heterogeneous and longitudinal electronic health record (EHR) data.

By Chantal Pellegrini, Ege \"Ozsoy, David Bani-Harouni, Matthias Keicher, Nassir Navab

nlp multimodal

arXiv Machine Learning

Jul 14

Multimodal Routing for Interpretable, Robust, and Auditable Clinical Prediction

arXiv:2607. 09982v1 Announce Type: new Abstract: Electronic health record (EHR) data are inherently multimodal, and leveraging multiple modalities can improve predictive performance.

By Nikkie Hooman, Zhongjie Wu, Eric C. Larson, Mehak Gupta

multimodal safety

arXiv Computer Vision

Oct 7

MedHorizon: Towards Long-context Medical Video Understanding in the Wild

arXiv:2605.06537v2 Announce Type: replace Abstract: Medical multimodal large language models (MLLMs) have advanced image understanding and short-video analysis, but real clinical review often require...

By Bodong Du, Bowen Liu, Yang Yu, Xinpeng Ding, Zhiheng Wu, Shuning Wang, Shuo Nie, Naiming Liu, Qifeng Chen, Yangqiu Song, Xiaomeng Li