← Back to all news
arXiv Machine Learning By Melika Shirian, Kianoosh Vadaei, Kian Majlessi, Audrina Ebrahimi, Peyman Adibi, Hossein Karshenas

PrismSSL: One Interface, Many Modalities; A Single-Interface Library for Multimodal Self-Supervised Learning

Read the original on arXiv Machine Learning →

PrismSSL is a Python library that consolidates leading self‑supervised learning methods for audio, vision, graphs, and cross‑modal tasks into a single, modular codebase. It allows users to install, configure, and run pretext training with minimal code, reproduce compact benchmarks, and extend the framework with new modalities or methods via clean trainer and dataset abstractions. The library, available on PyPI under an MIT license, integrates with HuggingFace Transformers and offers features such as distributed PyTorch training, Optuna hyperparameter search, LoRA fine‑tuning, animated embedding visualizations, Weights & Biases logging, and a Flask‑based graphical dashboard for easy pipeline configuration.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 15

PrismaCoder: Pushing the Boundaries of Open Multimodal Code Intelligence through Decomposed Rubric Synthesis

arXiv:2610.20538v1 Announce Type: cross Abstract: Multimodal code intelligence connects the symbolic world of programs with the visual world their execution creates. Interest in this capability is bu...

By Qiushi Sun, Jingyang Gong, Yang Liu, Qiaosheng Chen, Zhangyue Yin, Zixian Huang, Hanxu Hu, Jing Xiong, Fangzhi Xu, Haochen Tan, Haoli Bai, Lifeng Shang, Xiaoguang Li, Ben Kao, Qipeng Guo, Lingpeng Kong, Jiahui Gao
reinforcement-learningmultimodaldata-engineering
More like this →
arXiv AI
Oct 9

UniData: Universal Multimodal Instruction Generation Pipeline

UniData is a universal multimodal instruction generation pipeline that transforms simple user requirements into multi‑round, multimodal instructions. It expands user inputs into diverse events, uses an any‑to‑any large model for instruction generation, and refines the output by correcting irrelevant or redundant inference flows. The authors also introduce UniDataset, a 20,000‑entry dataset spanning nine modalities, and show that UniData achieves state‑of‑the‑art data quality and improves other multimodal models’ understanding and generation capabilities.

By Jiaqi Tang, Yi-Feng Wu, Yuting Zhang, Hao Lu, Bowen Fu, Qing-Guo Chen, Xiaogang Xu, Yuwei Hu, Shiyin Lu, Wei Wei, Lei Zhang, Zhao Xu, Weihua Luo, Qifeng Chen, Ying-Cong Chen
llmsmultimodalbenchmarksdata-engineering
More like this →
arXiv Computer Vision
Oct 16

V-GIFT: Boosting Visual Instruction Tuning with Self-Supervised Guidance

The paper introduces V‑GIFT, a lightweight method that enhances visual instruction tuning for multimodal large language models by incorporating a small set of visually grounded self‑supervised tasks expressed as natural language instructions. These tasks—such as rotation prediction, color matching, and cross‑view correspondence—are reformulated into image‑instruction‑response triplets, providing supervision that cannot be solved without visual evidence. The approach requires no human annotations, architectural changes, or extra training stages, and consistently improves vision‑centric performance across various models, regimes, and benchmarks when only 3‑10% of the training data is augmented with these instructions.

By Sophia Sirko-Galouchenko, Monika Wysoczanska, Andrei Bursuc, Nicolas Thome, Spyros Gidaris
llmsfine-tuningmultimodalbenchmarks
More like this →
arXiv Computer Vision
Sep 28

A Controlled Study of Self-Supervised Image and Video Pretraining under Limited Resources

The paper reports a controlled study of self‑supervised learning (SSL) objectives for image and video pretraining under limited data, architecture, and compute budgets. It compares contrastive, reconstruction, feature‑prediction, and diffusion methods, finding that DINOv2‑style pretraining delivers the best overall performance. Combining DINOv2 with video SSL objectives such as VideoMAE improves image classification and segmentation but harms video tracking and camera‑pose estimation, highlighting a trade‑off between semantic and geometric learning.

By Brun\'o B. Englert, Gijs Dubbelman
diffusioncomputer-vision
More like this →