← Back to all news
arXiv Machine Learning By Wei Wang, Wei Jiang, Ziran Liu

CHASE: Channel-Aligned Structure Exploitation for Geometry-Aware Model Engineering

Read the original on arXiv Machine Learning →

The paper introduces CHASE, a framework that leverages Geometric and Spectral Alignment (GSA) to inform practical model design. GSA characterizes trained networks via spectral concentration, channel alignment, support structure, and singular basis changes, and CHASE applies these insights to six applications including parameter‑efficient fine‑tuning, structured pruning, and model merging. New methods—CAGA, SAKV, and CAPS—use GSA to share attention heads, compress KV caches, and prune structured outputs, demonstrating improved performance over baselines.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 3

Output-aware Residual Stream Pruning for Large Language Models

The paper proposes a sensitivity‑aware residual‑stream pruning method for large language models that goes beyond minimizing activation reconstruction error. By using a second‑order approximation of output KL divergence, the authors derive a spectral upper bound that selects pruning subspaces based on both activation covariance and output sensitivity, enabling efficient eigendecomposition. Experiments on instruction‑tuned language models show that this approach consistently reduces calibration KL divergence, improves perplexity, and enhances downstream task performance across various compression levels.

By Chayne Thrash, Kevin Chen, Soheil Kolouri
llmsefficiency
More like this →