arXiv Machine Learning

nnInteractive Agent: General-Purpose AI Agents as 3D Image Annotators

The paper introduces nnInteractive Agent, an open‑source system that lets general‑purpose AI agents perform 3D image annotation tasks by interacting with the nnInteractive segmentation model. Users issue plain‑language requests, and the agent automatically identifies targets, places prompts, evaluates and corrects the resulting volumetric masks, and can even generate code for additional processing. On a diverse benchmark of CT, MRI, PET, micro‑CT, and insect scans, agents such as Claude Code and Codex achieve mean Dice scores of 0.81–0.84 from the request alone, matching near‑oracle performance when given target locations.

arXiv AI
Sep 24

NV-Reason-CT: 3D Visual Language Model for CT Analysis

NV-Reason-CT is a generative vision‑language model designed for chest and abdominal CT analysis that preserves native 3D visual encoding and incorporates radiologist‑guided reasoning. The system couples a 3D vision transformer with a language model, feeding all visual tokens and their 3D coordinates directly into language decoding to maintain volumetric spatial information. Trained on a curated corpus of about 550,000 multimodal instruction examples, the model supports abnormality classification, report generation, and interactive reasoning, achieving strong performance on CT benchmarks and reducing expert interpretation time by 50%.

By Andriy Myronenko, Dong Yang, Yucheng Tang, Baris Turkbey, Benjamin Simon, Stephanie Harmon, Rikhil Makwana, Mariam Aboian, Sena Azamat, Ibrahim Ethem Hamamci, Sezgin Er, Bjoern Menze, Marc Edgar, Yufan He, Pengfei Guo, Daguang Xu
arXiv Machine Learning
Aug 31

Do Medical Vision Models Reason About Anatomy? Probing the Spatial Inductive Biases of Learned Visual Representations

The paper introduces SPAR‑Bench, a set of eight probes designed to test whether medical vision models can reason about anatomy in abdominal CT scans. Experiments across five architectures and three foundation models—both frozen and fine‑tuned—show that while models can recall canonical organ locations, they fail to perform relational reasoning or spatial comparisons within a patient, even under zero‑shot transfer. The study also demonstrates that pooled probing underestimates a model’s relational capabilities and that open‑weight multimodal large language models perform poorly on these tasks.

By Naren Akash, Neeraja Ramanan
arXiv Computer Vision
Aug 28

DALE-CT: Depth-Aware 2D Slice Encoders Learn an Anatomical World Model of Chest CT

DALE-CT introduces depth‑aware 2D slice encoders that learn an anatomical world model of chest CT scans without 3D or positional supervision. By sampling self‑supervised views across a physical $z$‑axis slab, the encoder captures how anatomy changes between neighboring slices, enabling it to recover slice ordering and distinguish slices by anatomy alone. The model, trained on a large 287k‑scan corpus, achieves state‑of‑the‑art performance on CT‑RATE and is released with full code and evaluation tools.

By Evan W. Damron, Mahmut S. Gokmen, Mitchell A. Klusty, Caroline N. Leach, Emily B. Collier, V. K. Cody Bumgardner