← Back to all news
arXiv Machine Learning By Payam Abdisarabshali, Fardis Nadimi, Kasra Borazjani, Naji Khosravan, Richeng Jin, Seyyedali Hosseinalipour

BLEND: Balancing Personalization vs. Generalization in Federated Vision-Language Models

Read the original on arXiv Machine Learning →

BLEND is a framework for federated vision‑language models that balances personalization and generalization. It uses selective fine‑tuning with global vision and text adapters shared across clients and a personalized vision projection adapter for each client. A novel loss function blends outputs from global and personalized adapters, while an anchor design in the projection heads promotes generalization, leading to superior performance over existing baselines.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Computer Vision
Oct 2

Beyond Domain-Level Adaptation: Margin-Oriented Semantic-Appearance Interaction Correction for Personalized Federated Vision-Language Models

The paper introduces MOSAIC, a method for personalized federated vision‑language models that addresses domain heterogeneity by focusing on class‑specific cross‑domain residuals. It constructs a decision‑aware harmfulness score to identify residuals that hurt image‑text decision margins, then uses a low‑rank residual adapter with shared class factors and private domain factors, an image‑conditioned gate, and harmful‑pair‑aware reweighting to refine local updates. Experiments on Office31, OfficeHome, and DomainNet100 show consistent improvements in macro‑client top‑1 accuracy across various domain‑shift scenarios.

By Wentao Yue, Qingyu Mao, Tianyou Lai, Ahmed M. Abdelmoniem, Qilei Li
llmsfine-tuningmultimodal
More like this →
arXiv Machine Learning
3d ago

Is High Local Accuracy Enough? Geometric Consensus with Support-Adaptive Statistical Personalization for Federated Learning

The paper introduces FedGRC, a federated learning framework that enhances both global recognition and personalization for heterogeneous clients. It combines Consensus‑Guided Representation Learning, Multilayer Metric Synthesis, and Support‑Adaptive Shrinkage Personalization to address failures on rare or unseen classes. Experiments on CIFAR‑10, CIFAR‑100, and the FedRS dataset show that FedGRC outperforms 21 baseline methods, improving global accuracy and personalization floors by over 2% in several settings.

By Junyu Lin, Bosong Qi, Pengcheng Shi, Jiayuan Li
benchmarks
More like this →
arXiv Computer Vision
Oct 19

FedRAPP: Federated Test-Time Prompt Tuning for Vision-Language Models with Reliability-Aware Prompt Personalization

FedRAPP introduces a personalized aggregation framework for federated test‑time prompt tuning of vision‑language models, enabling distributed clients to adapt soft prompts on unlabeled test streams without source data. The method incorporates reliability‑aware client assessment, server‑side exponential moving‑average prompt memory, and similarity‑guided peer aggregation to selectively combine information from reliable and compatible clients. Experiments on nine CLIP benchmark datasets and ImageNet variants show that FedRAPP consistently outperforms standard federated aggregation baselines under heterogeneous client distributions while preserving prompt‑only adaptation efficiency.

By Syed Md. Ahnaf Hasan, Md Akil Raihan Iftee, Sajib Mistry, Amin Ahsan Ali, AKM Mahbubur Rahman
llmsmultimodalbenchmarks
More like this →