arXiv Computer Vision
NitroFusion2: Bringing Adversarial Loss to the Text Stream of Multimodal Diffusion Transformers
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Computation and Language
Aug 28
Visual Information-Guided Parallel Decoding for Diffusion Multimodal Large Language Models
Visual Information-Guided Parallel Decoding for Diffusion Multimodal Large Language Models introduces the VIG‑Sampler, a method that prioritizes tokens for decoding based on their attention to image tokens and penalizes redundancy in image‑attention distributions. The approach aims to improve the quality of multimodal generation by selecting more informative tokens during diffusion decoding. Experiments on seven captioning and VQA benchmarks with three open‑source dMLLMs show that VIG‑Sampler outperforms the Info‑Gain Sampler by an average of 19.3 CIDEr points and achieves better COCO Caption results using only half as many decoding steps.
arXiv AI
Jun 11
Diffusion-based Cumulative Adversarial Purification for Vision Language Models
arXiv:2506. 03933v2 Announce Type: replace-cross Abstract: Vision Language Models (VLMs) have shown remarkable capabilities in multimodal understanding, yet their susceptibility to adversarial perturbations poses a significant threat to their reliability in real-world applications.
arXiv Computer Vision
Sep 7
PAPT++: Risk-Aware Adversarial Tuning and Generation for Single Domain Generalization
PAPT++ is a risk‑aware adversarial generation‑training framework designed to improve single domain generalization. It learns diverse semantic reference images per class and uses them as denoising targets in classifier‑guided diffusion synthesis, thereby generating challenging yet semantically consistent samples. These samples are iteratively combined with source data to update the classifier, progressively exposing it to difficult variations and enhancing generalization performance on standard benchmarks.
arXiv AI
Jun 10
Improving Adversarial Transferability on Vision-Language Pre-training Models via Surrogate-Specific Bias Correction
arXiv:2606. 10571v1 Announce Type: cross Abstract: Adversarial examples reveal vulnerabilities in Vision-Language Pre-training (VLP) models and provide insights for improving robustness.
arXiv AI
Aug 5
Test-Time Scaling for Safe Text-Guided Image Generation via Intermediate Clean Estimates
arXiv:2608. 03284v1 Announce Type: cross Abstract: Ensuring safety and policy compliance in text-to-image diffusion models remains a critical challenge, as benign or adversarial prompts can often elicit prohibited content, e.
arXiv Computer Vision
Oct 21
Real In, Real Out: What If We Only Use Real Data for Scene Text Editing?
arXiv:2610.27592v1 Announce Type: new Abstract: Aiming to modify textual content while preserving the original style, Scene Text Editing (STE) has significant practical value in many applications. ST...