← Back to all news
arXiv AI By Omanshu Thapliyal

SSM Adapters via Hankel Reduced-order Modeling: Injection Site Determines Task Suitability in Long-Context Fine-Tuning

Read the original on arXiv AI →

arXiv:2606. 26290v1 Announce Type: cross Abstract: While parameter-efficient fine-tuning (PEFT) typically targets attention projectors, its efficacy for tasks requiring sequential state accumulation remains under-explored.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Jun 25

ATMA: Length-Invariant Language Modeling via Polar Attention and Gated-Delta Compression Memory

arXiv:2606. 25156v1 Announce Type: new Abstract: Modern large language models based on softmax scaled-dot-product attention are constrained by their training sequence length: as the key-value sequence grows, softmax probability mass can dilute across a wider distribution, inducing activation shift and long-context performance collapse.

By Habibullah Akbar
llmsbenchmarks
More like this →
arXiv Machine Learning
Sep 18

Elastic Spectral State Space Models for Train-Once Budgeted Inference

Elastic Spectral State Space Models (ES-SSM) are a train‑once, export‑many sequence modeling framework that achieves elasticity by spectrally approximating the state‑space operator. The method builds on Hankel spectral filtering, using fixed spectral channels to represent long‑range token mixing and combining input‑adaptive gates with budget dropout to enable reliable deployment across different resource budgets. ES‑SSM is evaluated on byte‑level language modeling, Long Range Arena, Speech Commands V2, and offline reinforcement learning, showing that a single trained model can be truncated to competitive compact models while maintaining smooth quality‑cost curves across a wide range of truncation levels.

By Dachuan Song, Junyu Yin, Zechen Hu, Xuan Wang
llmsreinforcement-learningbenchmarks
More like this →