arXiv Machine Learning
GeoPair: Geometry-Preserving Cross-Layer Factorization for Training-Free Transformer Compression
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jun 2
Learning Fine-grained Parameter Sharing via Sparse Tensor Decomposition
arXiv:2411. 09816v5 Announce Type: replace Abstract: Large neural networks achieve state-of-the-art performance on many tasks, yet their sheer size hinders deployment on resource-constrained devices.
arXiv Machine Learning
Jul 3
BALF: Budgeted Activation-Aware Low-Rank Factorization for Fine-Tuning-Free Model Compression
arXiv:2509. 25136v3 Announce Type: replace Abstract: Activation-aware low-rank factorization techniques yield strong compression results but are generally confined to linear layers, while existing whitening-based theory typically makes an implicit full-rank assumption on activations.
arXiv Machine Learning
Sep 10
Dense Structural Compression of Transformers via Gauge-Correct Channel Removal
The paper introduces GaugeLasso, a method that applies symmetric group‑lasso penalties to transformer channels during training, enabling entire tensor slices to be zeroed out while maintaining dense tensors for GPU efficiency. By calibrating channel penalties based on inference utility per compute, the network self‑organizes into depth‑dependent structural profiles that can be dramatically smaller than the original architecture, achieving up to 255‑fold compression on a polynomial division task and outperforming hand‑designed baselines on language modeling and autoencoding benchmarks. The approach also accelerates training and reveals over‑provisioned axes that guide subsequent design iterations.
Hugging Face Trending Papers
Aug 26
CrossMambaTuning: Synergistic Spatial and Cross-Layer Adaptation for Machine Vision Compression
To reduce deployment cost and retraining overhead, adapting pretrained learned image compression (LIC) models to downstream machine vision tasks has attracted growing attention. However, existing meth...
arXiv Computer Vision
Aug 27
CrossMambaTuning: Synergistic Spatial and Cross-Layer Adaptation for Machine Vision Compression
CrossMambaTuning is a new framework that adapts pretrained learned image compression models to machine vision tasks with minimal retraining. It combines State Space Models with cross‑layer interaction, featuring a Mamba adapter that uses task‑specific prompts and multi‑scale branching, and a Scale‑Invariant Cross‑Layer Adapter (SICA) that shares parameters across scales to reduce redundancy. Experiments show that this approach achieves state‑of‑the‑art performance while cutting parameter overhead by 72% compared to existing methods.
arXiv AI
Aug 12
TACTICL: Task-Aware Compression of Tabular ICL Models
arXiv:2608. 10837v1 Announce Type: cross Abstract: The strong performance of foundation models for tabular tasks comes at substantial inference costs.