Hugging Face Trending Papers
Long-Context Demonstration Selection Using State Space Models
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
Sep 17
Long-Context Demonstration Selection Using State Space Models
The paper addresses the challenge of selecting demonstrations for long-context language model queries, where transformer inference costs grow quadratically with sequence length. It proposes two algorithms that distill transformer behavior into state space models (SSMs) with linear inference time, partitioning transformer layers into groups and estimating separate SSMs for each. The distilled SSMs achieve less than 0.7% approximation error, and in downstream tasks they reduce FLOPs by 14.2× while improving accuracy by 6.48% compared to baseline methods.
arXiv AI
Sep 1
DIP: Dynamic In-Context Planner For Diffusion Language Models
arXiv:2601.03199v2 Announce Type: replace-cross Abstract: Diffusion language models (DLMs) have shown strong potential for general natural language tasks with in-context examples. Existing In-Context...
arXiv Computation and Language
4d ago
Pretraining Latent Information Feedback Transformers with Teacher Supervision
arXiv:2609.38149v1 Announce Type: new Abstract: Transformer language models (LMs) are feed-forward: deep-layer representations are never fed back to shallower layers, and the only pathway for informa...
Hugging Face Trending Papers
Sep 23
Distilling Sequential Computation in Transformer Language Models
Transformer language models process sequences token by token in an autoregressive manner, making growing contexts increasingly expensive. Yet many adjacent token spans are highly predictable or freque...
Hugging Face Trending Papers
Sep 15
Large Language Models Develop Belief State Geometry In-Context
Large language models (LLMs) trained on next-token prediction exhibit remarkable in-context learning (ICL) abilities, yet the representations that support ICL remain poorly understood. We consider suc...
arXiv AI
2d ago
Capturing In-Context Learning Dynamics with Task Operators
arXiv:2610.01054v1 Announce Type: cross Abstract: In-context learning (ICL) enables language models to perform new tasks from demonstrations without weight updates. However, every ICL inference requi...