Hugging Face Trending Papers

Visual Anchoring in Diffusion: Multimodal Zero-Shot Skeleton Action Recognition

Zero-shot Skeleton Action Recognition (ZSAR) remains ambiguous when unseen actions share similar skeleton joint dynamics but differ in objects or scene context. RGB provides these missing cues, yet existing multimodal methods typically maintain independent skeleton and RGB scoring branches and fuse their outputs.

arXiv AI
Sep 3

InstEditSeg: Instruction-Driven Image Editing for Polyp and Skin Lesion Segmentation

InstEditSeg is a generative framework that treats medical segmentation as an instruction-driven image editing task. Instead of producing binary masks, it renders a color-coded overlay on the original image guided by textual instructions, leveraging latent diffusion models to align with natural image distributions and reduce domain gaps. The method incorporates a DINOv3 visual encoder and a multi-scale feature pyramid fused into the diffusion U‑Net, and uses a dual‑branch classifier‑free guidance strategy to lower inference cost, achieving competitive accuracy on polyp and skin lesion datasets while improving cross‑domain generalization and multi‑lesion segmentation.

By Ziquan Liu, Zhewei Zhu, Xuyang Shi
arXiv Computer Vision
6d ago

Timo: $\textbf{T}$aming Mult$\textbf{i}$modal Diffusion Transformer for Human $\textbf{Mo}$tion Generation

The paper introduces Timo, a kinematics-aware multimodal diffusion transformer designed for human motion generation. Timo employs fully shared multimodal attention, flow matching, and geometric/rotational-kinematics supervision to better coordinate articulated motion, and uses a two-stage curriculum to align motion with text captions. The authors also present a new benchmark of 40,025 clips from six datasets, showing that Timo outperforms state‑of‑the‑art methods, achieving a 40.8% relative improvement over Kimodo on average.

By Zhao Wang, Jiangtao Hu, Jack Yu, Tao Yu
arXiv AI
Aug 28

A Unified Conditional Flow for Motion Generation, Editing, and Intra-Structural Retargeting

The paper introduces a unified conditional-flow framework that integrates text-driven motion generation, semantic editing, and intra-structural retargeting into a single rectified-flow model. By treating editing as a change in semantic condition and retargeting as a change in skeletal condition, the approach eliminates fragmented pipelines and allows a single model to perform generation, zero‑shot editing, and zero‑shot retargeting on articulated 3D motion data. Experiments on SnapMoGen and a Mixamo subset demonstrate that the model can handle all three tasks without task‑specific fine‑tuning, preserving both motion semantics and skeletal structure.

By Junlin Li, Xinhao Song, Siqi Wang, Haibin Huang, Yili Zhao
arXiv AI
Sep 2

Semantic-Guided Multimodal Preprocessing for Vision Transformer-Based Clear Cell Renal Cell Carcinoma Grading

The paper introduces a semantic‑guided multimodal preprocessing technique that fuses nuclei classification maps with RGB histopathology images for Vision Transformer‑based grading of clear cell renal cell carcinoma. By concatenating classification map channels and applying multiplicative modulation, the method achieves a balanced accuracy of 0.916, markedly surpassing an RGB‑only baseline (0.707) and prior max‑voting approaches (0.427). Sensitivity analysis shows the 21‑percentage‑point improvement remains robust under simulated perturbations matching current nuclei classifier error rates, indicating effective use of imperfect nuclear‑level information.

By Fatemeh Javadian, Zhu Chen, Zahra Aminparast, Johannes Stegmaier
arXiv Computer Vision
Sep 11

UniH$^3$: Unifying Hierarchical Homogeneity and Heterogeneity for All-in-One Medical Image Restoration

UniH$^3$ is a new framework for all-in-one medical image restoration that unifies hierarchical homogeneity and heterogeneity. It introduces a Hierarchical Homogeneity Memory (H2M) module to distill and retrieve shared anatomical priors, and a Hierarchical Heterogeneity Balancer (H2B) to mitigate inter- and intra-task conflicts during training. Experiments on MedIR-2D-500K and MedIR-3D-3K show that UniH$^3$ achieves state‑of‑the‑art performance for both multi‑task and single‑task restoration.

By Zhiwen Yang, Jiayin Li, Chengyu Liu, Hui Zhang, Bingzheng Wei, Yan Xu