The paper introduces SPD-MetaFormer, an attention‑free architecture for decoding brain signals from small datasets. It shows that existing attention‑based models on the SPD manifold (MAtt and GBWAtt) learn nearly uniform weights, and that replacing learned weights with uniform ones does not degrade performance. SPD-MetaFormer uses uniformly weighted Fréchet aggregation with a geodesic residual and a shared spectral feedforward map, achieving competitive results on three EEG benchmarks while maintaining SPD‑valued token states until classification.
arXiv:2606. 25456v1 Announce Type: new Abstract: Brain-Computer Interface (BCI) based on electroencephalography (EEG) enables direct interaction between the brain and external environments and has significant applications in assistive technologies, medical rehabilitation, and entertainment.
arXiv:2606. 18587v1 Announce Type: cross Abstract: Decoder-only Transformers compute attention over the KV cache of preceding tokens.
The paper introduces SCAP (Stiefel Cross-Attention Pool), a routing layer that uses a pool of Stiefel filters combined via cross‑attention to create a sample‑specific bilinear map. SCAP addresses the capacity ceiling problem of standard ReEig nonlinearity in SPD networks, where a single filter cannot align multiple domains. Experiments on five cross‑domain EEG motor‑imagery datasets show SCAP improves balanced accuracy over fixed‑filter SPDNet and matches or surpasses three domain‑adaptive baselines on most datasets.
arXiv:2605. 18848v3 Announce Type: replace Abstract: This paper introduces Exact Linear Attention (ELA), a mechanism that achieves linear computational complexity for Transformer attention by exploiting the exact decomposition property of kernel functions, thereby eliminating approximation error.
The paper introduces Adaptive Anisotropic Attention (AAA), a method that splits self‑attention into temporal and spatial paths for axis‑structured signals like EEG. A learned gate combines the two paths for each token, and the resulting AXON model outperforms dense attention baselines on six EEG tasks and shows transfer to audio spectrograms. The study demonstrates that aligning attention with the natural axes of structured data provides a beneficial inductive bias.
Rhamba is a region‑aware pretraining framework for resting‑state fMRI that combines anatomically guided masking with hybrid Attention‑Mamba architectures. The study pretrained models on the ABIDE dataset using three masking strategies (Any, Majority, Pure) and evaluated four architectural variants, finding that the Mamba‑Attention (MA) hybrid achieved the best average AUROC on downstream schizophrenia and ADHD classification tasks. Explainable AI via Integrated Gradients highlighted that performance depends on the interaction between masking strategy and architecture rather than a single dominant configuration.
The paper introduces Mahalanobis-Based Multi-Head Attention for Complex State Propagation (MHA‑CSP), a new attention mechanism that replaces the standard dot‑product with a Mahalanobis distance‑based RBF kernel. This approach enables infinite‑dimensional feature space attention without extra parameters, allows direct construction of Tree Attention via LogSumExp correction, and incorporates an attention meshing mechanism for cross‑head collaboration. Experiments show that with only 119K parameters and teacher forcing applied only at the final hidden state, MHA‑CSP outperforms Transformer and GCN baselines on long‑sequence state tracking tasks, demonstrating efficient structured reasoning.
arXiv:2608. 02070v2 Announce Type: replace-cross Abstract: Brain-computer interfaces (BCIs) have been widely used in motor rehabilitation, disease diagnosis, and other neural engineering scenarios.
arXiv:2608. 02070v1 Announce Type: cross Abstract: Brain-computer interfaces (BCIs) have been widely used in motor rehabilitation, disease diagnosis, and other neural engineering scenarios.
SPDAlign is an interpretable framework that aligns domain-specific EEG data by applying linear transformations on the Symmetric Positive Definite manifold, correcting mean differences and global rotations across domains using Wasserstein Procrustes. It leverages a generative model of EEG to recover distribution shifts caused by factors like head geometry, and demonstrates competitive performance on extensive public EEG datasets. The method is globally linear, enabling identification of frequency ranges, spatial patterns, and cross-subject variability without requiring labeled calibration data.
arXiv:2606. 29200v1 Announce Type: new Abstract: Multi-site functional MRI (fMRI) studies are essential for robust neuropsychiatric diagnosis yet suffer severe domain shifts from scanner heterogeneity, demographics, and site-specific acquisition protocols.
arXiv:2606. 01294v1 Announce Type: cross Abstract: Linear attention reduces the quadratic cost of softmax attention by maintaining a recurrent fast-weight state, but it consistently lags on in-context retrieval and long-context tasks.