← Back to all news
arXiv AI By Ziquan Liu, Zhewei Zhu, Xuyang Shi

InstEditSeg: Instruction-Driven Image Editing for Polyp and Skin Lesion Segmentation

Read the original on arXiv AI →

InstEditSeg is a generative framework that treats medical segmentation as an instruction-driven image editing task. Instead of producing binary masks, it renders a color-coded overlay on the original image guided by textual instructions, leveraging latent diffusion models to align with natural image distributions and reduce domain gaps. The method incorporates a DINOv3 visual encoder and a multi-scale feature pyramid fused into the diffusion U‑Net, and uses a dual‑branch classifier‑free guidance strategy to lower inference cost, achieving competitive accuracy on polyp and skin lesion datasets while improving cross‑domain generalization and multi‑lesion segmentation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computer Vision
Sep 11

BruNet: A Cross-Domain Transfer Framework for Bruise Segmentation

BruNet is a new cross‑domain transfer framework for automatic bruise segmentation that combines a ViT‑based visual encoder (either self‑supervised DINOv3 or pretrained LingBot‑Vision) with a SAM‑based mask decoder. The model is trained on the HAM10000 skin lesion dataset and evaluated on a separate bruise dataset without any fine‑tuning, achieving superior performance over CNN‑based models, state‑of‑the‑art segmentation models, ChatGPT‑4o/5‑assisted SAM2 zero‑shot baselines, and the medical‑oriented MedSAM. This work represents the first study to address pixel‑level localisation of bruises, demonstrating strong cross‑domain generalisation.

By Qiming Wang, Richard J. Motley, Ebube E. Obi, Xianfang Sun, Paul L. Rosin
llmscomputer-visionfine-tuningbenchmarks
More like this →
arXiv Machine Learning
Aug 27

Unsupervised Anatomical Feature Learning via Diffusion Models: Enhanced Medical Image Segmentation with Denoising Diffusion Probabilistic Models

The paper introduces an unsupervised approach to medical image segmentation by training a Denoising Diffusion Probabilistic Model (DDPM) on 21 unlabeled abdominal CT scans to learn anatomical features. The encoder weights from the DDPM are transferred to a U‑Net for downstream segmentation on the BTCV multi‑organ dataset, resulting in a significant Dice score improvement for liver segmentation from 0.75 to 0.93. In low‑data regimes, diffusion‑pretrained models retain robust performance, achieving high Dice scores even with only 10% of labeled data.

By Akshat G, Divyansh Gupta, Shaleen Bhatnagar, Shilpa Ankalaki, Tusar Kanti Mishra
diffusioncomputer-visionfine-tuning
More like this →