← Back to all news
arXiv Machine Learning By Haengbok Chung, SunGyu Kim, Joo hyun Lee, Sangjin Bae, Min Jeong Cho, Minseok Suh, Jae Sung Lee

A Specialized Large Multimodal Model for Interpreting PET/CT in Head and Neck Cancer

Read the original on arXiv Machine Learning →

A specialized large multimodal model, LLaVA‑NeXT, was fine‑tuned on a curated two‑level curriculum of PET/CT image‑conversation pairs to interpret head and neck cancer scans. In external validation across four institutions, the model achieved high ROUGE and similarity scores and outperformed generalist models such as ChatGPT, with primary tumor classification accuracy of 83.14% internally and 69.03% externally. The study demonstrates that domain‑specific LMMs can provide fast, accurate diagnostic support for PET/CT imaging.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Aug 19

Comprehensive framework for evaluation of deep neural networks in detection and quantification of lymphoma from PET/CT images: clinical insights, pitfalls, and observer agreement analyses

This study presents a clinically relevant framework for evaluating deep neural networks that segment lymphoma lesions in PET/CT images, addressing gaps such as out‑of‑distribution testing and comparison with expert annotators. Using 611 multi‑institutional cases, the authors assess four networks (ResUNet, SegResNet, DynUNet, SwinUNETR) with lesion‑specific metrics, detection criteria, and metabolic‑characteristic‑based thresholds, finding that models perform best on large, intense lesions. The work also demonstrates that network errors mirror those of physicians, highlighting shared challenges with small, faint lesions.

By Shadab Ahamed, Yixi Xu, Sara Kurkowska, Claire Gowdy, Joo H. O, Ingrid Bloise, Don Wilson, Patrick Martineau, Fran\c{c}ois B\'enard, Fereshteh Yousefirizi, Rahul Dodhia, Juan M. Lavista, William B. Weeks, Carlos F. Uribe, Arman Rahmim
computer-vision
More like this →
arXiv Computer Vision
Aug 21

MUST-PET: MUltimodal Self-supervised learning across Tracers for whole-body PET/CT-based lesion segmentation

arXiv:2608. 19666v1 Announce Type: new Abstract: Deep learning-based whole-body PET-CT lesion segmentation can support cancer staging, treatment planning, and response assessment, but generalization is limited by scarce annotations and domain shifts.

By Bashirul Azam Biswas, Amartya Bhattacharya, Biratal Raj Wagle, Matthew E. Maeder, James B. Yu, Indrani Bhattacharya
computer-visionfine-tuningmultimodal
More like this →
Hugging Face Trending Papers
Jul 28

Comparing the Performance of Foundation Model Derived Embeddings with Traditional Approaches for Distant Metastasis Prediction in Head and Neck Cancer

Background: Early prediction of distant metastasis (DM) risk in head and neck cancer (HNC) can enable timely interventions that may improve treatment outcomes. Many current machine learning methods rely on prior knowledge of the region of interest such as tumor segmentations, which require expert knowledge, is time-consuming and introduces user-dependent variability.

rag
More like this →