← Back to all news
arXiv AI By Ziquan Liu, Zhewei Zhu, Xuyang Shi

InstEditSeg: Instruction-Driven Image Editing for Polyp and Skin Lesion Segmentation

Read the original on arXiv AI →

InstEditSeg is a generative framework that treats medical segmentation as an instruction-driven image editing task. Instead of producing binary masks, it renders a color-coded overlay on the original image guided by textual instructions, leveraging latent diffusion models to align with natural image distributions and reduce domain gaps. The method incorporates a DINOv3 visual encoder and a multi-scale feature pyramid fused into the diffusion U‑Net, and uses a dual‑branch classifier‑free guidance strategy to lower inference cost, achieving competitive accuracy on polyp and skin lesion datasets while improving cross‑domain generalization and multi‑lesion segmentation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Aug 27

Unsupervised Anatomical Feature Learning via Diffusion Models: Enhanced Medical Image Segmentation with Denoising Diffusion Probabilistic Models

The paper introduces an unsupervised approach to medical image segmentation by training a Denoising Diffusion Probabilistic Model (DDPM) on 21 unlabeled abdominal CT scans to learn anatomical features. The encoder weights from the DDPM are transferred to a U‑Net for downstream segmentation on the BTCV multi‑organ dataset, resulting in a significant Dice score improvement for liver segmentation from 0.75 to 0.93. In low‑data regimes, diffusion‑pretrained models retain robust performance, achieving high Dice scores even with only 10% of labeled data.

By Akshat G, Divyansh Gupta, Shaleen Bhatnagar, Shilpa Ankalaki, Tusar Kanti Mishra
diffusioncomputer-visionfine-tuning
More like this →