arXiv Machine Learning

Spectral Rank Certification for Foundation Model Adapters

arXiv:2608. 15351v1 Announce Type: new Abstract: Nominal LoRA rank is a design parameter; calibrated spectral evidence is a separate inferential quantity.

arXiv AI

Jun 2

UR-JEPA: Uniform Rectifiability as a Regularizer for Joint-Embedding Predictive Architectures

arXiv:2606. 01443v1 Announce Type: cross Abstract: A central difficulty in training Joint-Embedding Predictive Architectures (JEPAs) is preventing representation collapse.

By Triet M. Le

rag

arXiv AI

Jun 15

Beyond LoRA: Is Sparsity-Induced Adaptation Better?

arXiv:2606. 13767v1 Announce Type: cross Abstract: Low-rank adaptation (LoRA) and its variants provide a memory- and compute-efficient alternative to full fine-tuning of pre-trained models.

By Elijah Cadenhead, Cristian McGee, Xin Li, El Houcine Bergou, Aritra Dutta

fine-tuning efficiency

arXiv Machine Learning

Jul 21

De-floored Principal Component Regression: When Rank Selection Alone Is Insufficient for Prediction

arXiv:2607. 16638v1 Announce Type: cross Abstract: Principal component regression (PCR) regularizes high-dimensional prediction by choosing a spectral cutoff, but rank selection cannot correct systematic inflation of the retained empirical eigenvalues.

By Peng Zhao

safety

arXiv AI

Jun 2

Finer Parameter Steps for Low-Rank PEFT: A Controlled Study with CP Tensor Adapters

arXiv:2606. 00428v1 Announce Type: cross Abstract: Low-rank adapters are usually compared by sweeping a small set of ranks, but the rank also fixes the resolution of the parameter budget.

By Xinjue Wang, Xiuheng Wang, Yejun Zhang, Sergiy A. Vorobyov, Esa Ollila, Zhi-Yong Wang

fine-tuning

arXiv Machine Learning

Jul 31

Tight Sample Complexity for Low-Rank Adaptation: Matching Bounds and Rank Selection

arXiv:2607. 27680v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) has become the standard mechanism for fine-tuning large pretrained models, yet its statistical properties remain only partially understood.

By Arunan J

fine-tuning benchmarks

arXiv Machine Learning

Jun 9

Shortcuts in the Tail: Debiasing via Post-Hoc Spectral Compression of Fine-Tuning Updates

arXiv:2606. 07596v1 Announce Type: new Abstract: Fine-tuning often introduces spurious correlations alongside task knowledge, causing systematic failures on underrepresented groups.

By Edward Sun, Dmitrii Troitskii

fine-tuning benchmarks

arXiv Machine Learning

Jul 7

SAD-LoRA: Spectral Alignment for Low-Rank Knowledge Distillation

arXiv:2607. 04306v1 Announce Type: new Abstract: Distilling a fine-tuned teacher into a LoRA-adapted student is a standard recipe for parameter-efficient compression, but output-level KD does not explicitly control which rank-$r$ weight subspace the adapter occupies.

By Omer Tariq, Syed Muhammad Raza, Jeongbae Son

fine-tuning efficiency safety

arXiv AI

Jul 7

Incompressible Knowledge Probes: Estimating Black-Box LLM Parameter Counts via Factual Capacity

arXiv:2604. 24827v2 Announce Type: replace-cross Abstract: Closed-source frontier labs do not disclose parameter counts.

By Bojie Li

llms benchmarks safety

arXiv Machine Learning

Jun 5

The Evaluation Blind Spot: A Stereological Theory of Benchmark Coverage for Large Language Models

arXiv:2606. 05169v1 Announce Type: new Abstract: We give a stereological theory of LLM benchmark coverage.

By Jason Z Wang

llms benchmarks

arXiv Machine Learning

Jul 15

The Spectrum Is Not Enough: When Context Helps Time-Series Forecasting

arXiv:2607. 13006v1 Announce Type: new Abstract: A growing family of indices scores how predictable a series is from its spectrum.

By Mert Onur Cakiroglu, Mehmet Dalkilic, Hasan Kurban

benchmarks

arXiv Machine Learning

Jun 3

Compress then Merge: From Multiple LoRAs into One Low-Rank Adapter

arXiv:2606. 03723v1 Announce Type: new Abstract: Low-rank adaptation (LoRA) enables parameter-efficient specialization of foundation models, but the proliferation of task-specific adapters fragments capabilities across many adapters, complicating reuse and deployment.

By Zhengbao He, Ruiqi Ding, Zhehao Huang, Ruikai Yang, Tao Li, Xiaolin Huang

fine-tuning

arXiv Machine Learning

Aug 12

Iterative Erasure Count Is Not an Affine-Invariant Concept Dimension

arXiv:2608. 10566v1 Announce Type: cross Abstract: How many directions does a neural representation use to encode a concept?

By Tingan Jin, Shuhang Dong, Haosong Li, Chung-Hsien Chou

Related stories