← Back to all news
arXiv Computer Vision By Siyuan Liu, Xuze Zhang, Yongshun Wang, Licong Pan, Hang Liu, Huihui Li

Bridging Modalities and Tasks: A Unified Hierarchical ViT for SAR-to-Optical Translation and Semantic Segmentation

Read the original on arXiv Computer Vision →

The paper introduces BMT, a unified hierarchical Vision Transformer that jointly performs SAR-to-optical image translation and semantic segmentation. It incorporates a LocalViTBlock, an enhanced output module, a ControlNet-style conditional injection, and a bounded Kendall uncertainty weighting scheme to balance the two tasks. Experiments on paired and unpaired datasets demonstrate competitive performance in both translation quality and segmentation accuracy.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv Computer Vision
Sep 25

OptiSAR-Net++: A Large-Scale Benchmark and Transformer-Free Framework for Cross-Domain Remote Sensing Visual Grounding

OptiSAR-Net++ introduces a new cross‑domain remote sensing visual grounding task (CD‑RSVG) and the first large‑scale benchmark dataset, OptSAR‑RSVG. The framework replaces Transformer decoding with a CLIP‑based contrastive approach, employing a patch‑level Low‑Rank Adaptation Mixture of Experts for efficient cross‑domain feature decoupling and a text‑guided dual‑gate fusion module for improved semantic‑visual alignment. Experiments show state‑of‑the‑art performance on OptSAR‑RSVG and DIOR‑RSVG, with notable gains in localization accuracy and computational efficiency.

By Xiaoyu Tang, Jun Dong, Jintao Cheng, Rui Fan
llmsmultimodalbenchmarkssafety
More like this →
Hugging Face Trending Papers
Sep 2

Lightweight Adaptation of General-Purpose VLMs for Multispectral and SAR Image Understanding

The paper demonstrates a lightweight method to adapt general-purpose vision‑language models (VLMs) for multispectral and synthetic aperture radar (SAR) image understanding. By rendering each observation as five optical views and one SAR view, naming them in the prompt, and applying LoRA to the language network and selected visual transformer blocks, the authors enable VLMs to process band composites, spectral indices, and radar backscatter without retraining a new foundation model. On a balanced six‑class land‑cover benchmark from BigEarthNet‑v2, the adapted Qwen3‑VL achieves a micro F1 score of 0.8275, and the same protocol improves four other VLMs and transfers to flood verification and captioning tasks.

llmsfine-tuningmultimodalbenchmarks
More like this →
arXiv AI
Sep 2

SARTM: Segment Any RGB Thermal Model with Language aided Distillation

The paper introduces SARTM, a framework that adapts the Segment Anything Model (SAM) for RGB‑thermal (RGB‑T) semantic segmentation. It fine‑tunes SAM with LoRA layers, incorporates language guidance, and employs a Cross‑Modal Knowledge Distillation module to bridge modality gaps. The approach also modifies the segmentation head and adds an auxiliary semantic head, achieving superior performance on MFNET, PST900, and FMB benchmarks.

By Dong Xing, Jinhe Zhang, Hang Yang, Yuqing Wang
computer-visionfine-tuningefficiencymultimodalbenchmarks
More like this →
arXiv Computer Vision
Oct 8

SAREO-FM: Decoupled Semantic Supervision for SAR-EO Foundation Models

SAREO‑FM is a foundation model that separates semantic supervision from reconstruction supervision in synthetic aperture radar (SAR) and electro‑optical (EO) imagery. It uses modality tokens for masked reconstruction and learnable semantic queries guided by a pretrained vision foundation model to capture scene content. Trained on the million‑scale SAR‑1M corpus, the model shows strong unimodal transfer for SAR and EO alone and significant improvements when both modalities are combined.

By Jeonghyeok Do, Munchurl Kim
More like this →