arXiv Statistics ML
Gromov-Wasserstein Distillation for Inductive Multi-View Embedding
Read the original on arXiv Statistics ML →The Flow has not summarised this story yet — read it at arXiv Statistics ML.
Related stories
arXiv Machine Learning
Aug 14
Multiview Representation Learning via Distributed Joint Latent Space Structuring
arXiv:2504. 18455v2 Announce Type: replace-cross Abstract: We study distributed multiview representation learning, a problem in which $K$ clients each observe a distinct but possibly statistically correlated view.
arXiv Computer Vision
Aug 31
Relational Knowledge Distillation Brings DNN Representations Close Enough to Humans to Be Aligned Without Supervision
The study investigates whether transferring relational structure from human mental representations to deep neural networks (DNNs) can improve fine‑grained alignment between the two. Using unsupervised Gromov‑Wasserstein optimal transport, the authors show that fine‑tuning pre‑trained DNNs with Relational Knowledge Distillation (RKD) brings the networks close enough to human representations to align at the individual‑object level on a test set of concepts not seen during training. The improvement is driven mainly by a more human‑like global structure of category distances, while local nearest‑neighbor overlap remains largely unchanged.
arXiv Computer Vision
Sep 11
CLIP-RD: Relational Distillation for Efficient CLIP Knowledge Distillation
CLIP-RD introduces a relational distillation framework for efficient CLIP knowledge distillation, featuring Vertical Relational Distillation (VRD) and Cross Relational Distillation (XRD). VRD aligns intra‑modal similarity distributions between teacher and student, while XRD aligns cross‑modal similarity distributions to enforce bidirectional symmetry. This joint modeling of multidirectional relational structures improves the student’s embedding geometry, yielding a 1.8%p performance gain over CLIP‑KD across various architectures, tasks, and corruption settings with minimal training‑time overhead.
arXiv Machine Learning
Aug 20
Pretraining Reusable Inference Across Views with Synthetic Task Priors
The paper introduces SIMPLE, a prior‑fitted multi‑view in‑context learner that learns a reusable, task‑conditioned inference procedure instead of a fixed fusion function. By generating synthetic task priors in embedding space, SIMPLE can handle diverse view configurations, class structures, and missingness patterns. Experiments on multi‑view and multi‑omics benchmarks show that a frozen SIMPLE model performs competitively, and lightweight adapter calibration further improves performance across most datasets.
arXiv Computer Vision
4d ago
Spatial-OPSD: Self-Improving Spatial Reasoning via Label-Free Self-Distillation
Spatial-OPSD is a label‑free self‑improvement framework for vision‑language models that leverages spatial priors such as depth, 3D relations, and camera geometry to provide dense token‑level supervision. During training, a privileged teacher uses these priors while the student learns from only the original visual‑language input, and a recursive round‑wise scheme allows repeated self‑improvement without moving the teacher. Across four VLM families, one round of Spatial‑OPSD improves the five‑benchmark average, and three rounds push a strong spatially specialized model to the open‑source frontier, achieving the highest average among open models and best results on three of five spatial reasoning benchmarks.
arXiv Machine Learning
Jun 25
Knowledge Cascade: Reverse Knowledge Distillation on Nonparametric Multivariate Functional Estimation
arXiv:2606. 25927v1 Announce Type: cross Abstract: As machine learning models and datasets continue to grow, developing complex models has become increasingly computationally demanding.