The paper investigates how the upper spectral tails of weight matrices in decoder‑only transformer language models influence reasoning behavior. By performing controlled interventions on the query–key product and comparing them to factor‑level surgeries, the authors find that edits targeting the spectral tail more strongly affect model performance across multiple checkpoints and reasoning benchmarks. The study also explores how inverse participation ratios predict accuracy transitions and shows that tail‑aware low‑rank adaptations converge faster than standard methods.
By Ibne Farabi Shihab, Sanjida Akhter, Md Najmus Swaqeeb, Abu Sa-Adat Mohamed Moon-Im Al Ahsan, Anuj Sharma
The paper introduces a coupled query‑key transformation that jointly evolves queries and keys via an invertible coupling before the standard dot‑product scoring in attention mechanisms. Implemented as a lightweight alternating affine map, the coupling is added on top of existing attention methods and preserves the original softmax and architecture. Experiments on WikiText‑103 show that coupling improves performance when combined with Differential Attention, query‑key normalization, and Multi‑Token Attention, especially at larger model scales, while its standalone benefit diminishes with size.
By Barak Gahtan, Alex M. Bronstein
arXiv:2607. 15047v1 Announce Type: cross Abstract: Mild Cognitive Impairment is a critical early stage of cognitive decline that frequently precedes Alzheimer's disease, yet its automated detection from neuropsychological drawing tests remains fundamentally constrained by data scarcity, class imbalance, and diagnostic ambiguity near clinical boundaries.
By Javad Khoramdel, Farhad Hoseyni, Amirhossein Nikoofard
arXiv:2607. 22367v1 Announce Type: new Abstract: Feature-attribution methods assign scores relating input variables to a model's output, but do not by themselves characterize how explicitly defined interaction operators compose across its intermediate layers.
By Umberto Biccari, Qian Huang, Enrique Zuazua
The study investigates whether attention heads in large language models that align with human EEG signals are causally involved in model computation. By ablating these brain‑aligned heads during a pattern‑completion task, the authors find that while such heads contribute to performance, their removal is less disruptive than removing heads selected by attribution patching. The research also distinguishes two families of brain‑aligned heads—novelty and repetition heads—highlighting that novelty heads track human attention but are less critical than random ablation, whereas repetition heads modestly aid performance and align with abstract‑pattern representations.
By Christopher Pinier, Gustaw Opie{\l}ka, Hannes Rosenbusch, Taylor Webb, Michael D. Nunez, Claire E. Stevenson
The paper introduces RefineICL, an attention‑gated, feed‑forward‑network‑free framework that refines representations in situ for tabular foundation models. By using support labels to guide episode‑specific updates, the method transfers learned corrections to unlabeled queries without altering model parameters, achieving state‑of‑the‑art performance on AMLB29 and TabArena benchmarks. Experiments and internal interventions demonstrate that intermediate support updates are essential for constructing task‑specific predictors in context.
By Tian Zhou, Beverly Jin, Linxiao Yang, Xue Wang, Wenwei Wang, Bingqing Peng, Mengni Ye, Jinjie Gu, Liang Sun
arXiv:2601. 11618v2 Announce Type: replace-cross Abstract: Geometric Attention (GA) specifies an attention layer by four independent inputs: a finite carrier (what indices are addressable), an evidence-kernel rule (how masked proto-scores and a link induce nonnegative weights), a probe family (which observables are treated as admissible), and an anchor/update rule (which representative kernel is selected and how it is applied).
By Luis Rosario Freytes
BioSync is a transformer-based model that fuses cardiac, neural, behavioral, and speech data from wearables and mobile devices into a continuous composite digital biomarker called the BioSync Index (BSI). The architecture uses multi-head self-attention on modality tokens and a linear branch for feature concatenation, inspired by latent-variable measurement theory. Evaluations on synthetic cohorts for cognitive decline and metabolic-autonomic conditions show BioSync achieving AUCs of 0.928 and 0.764 accuracy/F1 of 0.766, outperforming simple concatenation and other fusion strategies in most corruption scenarios.
By Seyed Mahmoud Sajjadi Mohammadabadi
The paper introduces PL‑SCEA, a method that reconfigures the attention mechanism of frozen Vision Foundation Models to better detect and localize anomalies in industrial images with few training examples. PL‑SCEA preserves the semantic context of pretrained query‑key attention while adding token‑adaptive self‑correlations over contextualized value features, then applies positive‑correlation filtering and power‑law reweighting to highlight task‑relevant relationships. The resulting features are fed into a lightweight variational autoencoder to produce reconstruction‑based anomaly scores, achieving competitive image‑level detection and strong pixel‑level localization on MVTec AD and VisA datasets.
By Xiaoyu Yang, Qixing Wu, Huixian Zhao, Changlong Jin
arXiv:2602. 23459v2 Announce Type: replace Abstract: Psychiatric questionnaires are highly context sensitive and often only weakly predict subsequent symptom severity, which makes the prognostic relationship difficult to learn.
By Eric V. Strobl
AOI-Net introduces a structural face AOI-guided Eye‑Gaze Track Network that jointly models short‑term temporal dynamics and AOI‑level structural organization for Autism Spectrum Disorder detection. The network uses a gating mechanism to adaptively combine complementary representations and incorporates class‑distribution‑aware learning to address the imbalance between ASD and typically developing participants. Experiments on a large clinical eye‑tracking database with over 1,300 participants demonstrate that AOI‑Net outperforms state‑of‑the‑art methods and offers interpretable gaze‑behavior modeling for scalable AI‑driven ASD screening.
By Zhanpei Huang, Binbin Sun, Jialiang Chen, Yiou Wang, Taochen Chen, Yuzhu Ji, Yiqun Zhang, Yiu-Ming Cheung
arXiv:2605. 24059v2 Announce Type: replace Abstract: We present a three-step recipe for identifying attention-head circuits in pretrained transformers.
By Yongzhong Xu