arXiv Machine Learning October 16, 2026 By Haiwen Yi, Xinyuan Song

ManifoldFlow: Spectrum-Learnable Stiefel Optimization via SPD Relaxation

Read the original on arXiv Machine Learning →

arXiv:2607. 04535v4 Announce Type: replace Abstract: Orthogonal and Stiefel constraints are widely used in neural network optimization to control weight geometry, improve conditioning, and stabilize signal propagation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

safety

arXiv Machine Learning

Jul 7

ManifoldFlow: SPD-Relaxed Stiefel Layers with Learnable Singular Spectrum

arXiv:2607. 04535v1 Announce Type: new Abstract: Orthogonal and Stiefel layers give neural weights exact spectral control, but they also impose a strong modeling constraint: all represented singular values are fixed at one.

By Haiwen Yi, Xinyuan Song

arXiv AI

Sep 28

Spectral-Sphere-Constrained Hyper-Connections

The paper introduces Spectral‑Sphere‑Constrained Hyper‑Connections (s²HC), a new method for controlling the residual matrices used in Hyper‑Connections (HC). Unlike previous doubly stochastic constraints that caused identity degeneration, expressivity bottlenecks, and parameterization inefficiencies, s²HC confines these matrices to a spectral norm sphere, restoring flexibility over subdominant spectra and eliminating unstable Sinkhorn‑Knopp iterations. This approach preserves training stability while allowing expressive, non‑degenerate residual matrices.

By Zhaoyi Liu, Haichuan Zhang, Ang Li

arXiv Machine Learning

Jul 3

Spectral Imbalance Causes Forgetting in Low-Rank Continual Adaptation

arXiv:2602. 00722v2 Announce Type: replace Abstract: Parameter-efficient continual learning aims to adapt pre-trained models to sequential tasks without forgetting previously acquired knowledge.

By Hao Gu, Mao-Lin Luo, Zi-Hao Zhou, Han-Chen Zhang, Min-Ling Zhang, Tong Wei

llms multimodal

arXiv Machine Learning

Jun 19

Spectral DPPs via NEPv: A Scalable Continuous Relaxation of Determinantal MAP for Diversity-Aware Data Selection

arXiv:2606. 19411v1 Announce Type: new Abstract: Selecting a small, diverse, high-quality subset from a massive pool of candidates is a recurring primitive in modern machine learning -- data curation and coreset selection for training and fine-tuning large models, active-learning batch acquisition, prompt and exemplar selection for in-context learning, retrieval diversification, and experimental design.

By Richard Yi Da Xu

llms rag fine-tuning benchmarks

arXiv AI

Jul 29

Retraction-Free Optimization over the Stiefel Manifold for the LoRA Fine-Tuning

arXiv:2607. 25299v1 Announce Type: cross Abstract: Optimization over the Stiefel manifold plays a significant role in various machine learning tasks.

By Yuan Zhang, Jiang Hu, Zhijian Lai, Lin Lin, Zaiwen Wen

llms fine-tuning benchmarks

arXiv Machine Learning

Aug 14

Manifold constrained steepest descent for smooth and closed-set optimization

arXiv:2601. 21487v2 Announce Type: replace-cross Abstract: We study minimization of smooth functions over feasible sets that have smooth embedded-manifold structure throughout or only on selected regions, using linear minimization oracles (LMOs) to determine search directions under user-chosen norms.

By Kaiwei Yang, Lexiao Lai

One email a morning, machine-written

Related stories

ManifoldFlow: SPD-Relaxed Stiefel Layers with Learnable Singular Spectrum

Spectral-Sphere-Constrained Hyper-Connections

Spectral Imbalance Causes Forgetting in Low-Rank Continual Adaptation

Spectral DPPs via NEPv: A Scalable Continuous Relaxation of Determinantal MAP for Diversity-Aware Data Selection

Retraction-Free Optimization over the Stiefel Manifold for the LoRA Fine-Tuning

Manifold constrained steepest descent for smooth and closed-set optimization