← Back to all news
arXiv Machine Learning By Quang-Duy Tran, Trung Le, Bao Duong, Phuoc Nguyen, Thin Nguyen

Geometry-Aware Bayesian Parameter-Efficient Fine-Tuning on the Stiefel Manifold via Stein Variational Gradient Descent

Read the original on arXiv Machine Learning →

The paper introduces a geometry-aware Bayesian fine‑tuning method that uses Stein variational gradient descent on the Stiefel manifold. By transporting low‑rank adapter matrices along this manifold, the approach preserves orthogonality constraints and yields multiple inference solutions, enabling uncertainty quantification. Experiments demonstrate improved model calibration and higher prediction accuracy compared to Euclidean‑space SVGD and related methods.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Jun 30

BaRA: Bayesian Adaptive Rank Allocation for Parameter-Efficient Fine-Tuning

arXiv:2606. 29184v1 Announce Type: new Abstract: While Low-rank adaptation (LoRA) enables highly efficient fine-tuning by constraining task-specific updates to fixed low-rank subspaces, this rigid design limits representational flexibility and often results in overconfident predictions and miscalibrated uncertainty, especially in low-data regimes.

By Zhibin Duan, Yuhong Wang, Jiahong Fu, Zongsheng Yue, Bo Chen, Zongben Xu
fine-tuningbenchmarks
More like this →
arXiv Machine Learning
Sep 14

Nonlinear Dimensionality Reduction Techniques for Bayesian Optimization

The paper investigates nonlinear dimensionality reduction for Bayesian optimisation (BO) by transforming high‑dimensional black‑box optimisation problems into a sequence of low‑dimensional latent‑space BO (LSBO) tasks. It extends earlier linear embedding approaches by using variational autoencoders (VAEs), deep metric loss, and adaptive retraining to better capture nonlinear structure, and couples LSBO with sequential domain reduction (SDR‑LSBO) to progressively narrow search domains. Experiments on GPU‑accelerated BoTorch with Matérn‑5/2 Gaussian‑process surrogates show that VAE‑based LSBO outperforms adaptive linear embeddings, and the authors provide a theoretical analysis of latent‑space error versus representation gap under PAC‑Bayes conditions.

By Luo Long, Coralia Cartis, Paz Fink Shustin
ragdiffusionbenchmarks
More like this →