arXiv:2604. 11613v4 Announce Type: replace-cross Abstract: Transformers can perform in-context classification from a few labeled examples, yet the inference-time algorithm remains opaque.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
arXiv:2607. 00479v1 Announce Type: new Abstract: Transformer-based large models have demonstrated remarkable generalization abilities across different tasks by leveraging a context-aware attention module for in-context learning.
arXiv:2608.22368v1 Announce Type: new Abstract: While linear attention is a compelling mechanism for high-resolution object detection due to its reduced cost for global token mixing, converting the S...
arXiv:2606. 27748v1 Announce Type: cross Abstract: Transformer models rely on attention mechanism to capture long-range dependencies but suffer from quadratic complexity, limiting their scalability to long sequences.
We present a theoretical framework to explain the emergence of inductive reasoning abilities in Transformer language models. While previous works on Transformer learning dynamics have so far been mostly tied to specific tasks, we study a generalized class of inductive tasks that unifies several synthetic tasks known in the literature, including in-context n-grams and multi-hop reasoning.
arXiv:2606. 06458v1 Announce Type: new Abstract: Multiple Instance Learning (MIL) addresses problems where supervision is available at the level of bags of instances and has been successfully applied in fields ranging from computational pathology to satellite imagery.
The paper introduces Neural Low-Degree Filtering (Neural LoFi), a stylized limit of gradient-based training that turns hierarchical feature learning into an explicit iterative spectral procedure. In this framework, each layer independently selects directions with maximal low-degree correlation to the label, providing a tractable surrogate for deep learning and a kernel-space interpretation. Experiments on fully connected and convolutional networks show that Neural LoFi outperforms lazy random-feature baselines, recovers meaningful structured filters, and aligns with early gradient-descent feature discovery on real datasets.