arXiv AI

CauseCollab: Causal Unified and Modality-Agnostic Network for Heterogeneous Collaborative Perception

CauseCollab is a causal unified and modality‑agnostic network designed to improve collaborative perception across heterogeneous sensor modalities. It disentangles semantic factors from modality‑specific confounders using causal metric learning and employs a context‑guided Unified Converter to maintain cross‑modal semantic consistency. The approach requires only minimal adapter training when adding new modalities and achieves state‑of‑the‑art results on the OPV2V and DAIR‑V2X datasets, especially in scenarios with large modality gaps.

arXiv Computer Vision
3d ago

HeteroPROMPT: A Real-time and Privacy-Preserving Heterogeneous Collaborative Perception Framework

HeteroPROMPT is a real‑time, privacy‑preserving framework for heterogeneous collaborative perception in autonomous systems. It aligns features from diverse sensors and models into a unified ego‑centric space using modular prompts and lightweight tuning, while keeping encoders and fusion stacks frozen. The system employs a metadata‑free autoencoder for modality classification and routing, achieving higher average precision on OPV2V‑H and V2XSet datasets with far fewer trainable parameters.

By Armin Maleki, Hayder Radha
arXiv Machine Learning
Aug 17

MedMix: Specialization-Consistent Federated Sparse MoEs under Modality Heterogeneity

arXiv:2608. 13911v1 Announce Type: new Abstract: Federated multimodal medical AI faces modality heterogeneity at both the client and sample levels: clients may systematically lack access to specific modality types, while individual records within the same client may contain different partial modality subsets.

By Adiba Orzikulova, Dong Min Kim, Jaehong Yoon, Sung-Ju Lee