← Back to all news
arXiv Machine Learning By Zhida Wang, Wei Lyu, Guo Yu, Sui Tang

SPD-MetaFormer is what you need for small-data brain decoding

Read the original on arXiv Machine Learning →

The paper introduces SPD-MetaFormer, an attention‑free architecture for decoding brain signals from small datasets. It shows that existing attention‑based models on the SPD manifold (MAtt and GBWAtt) learn nearly uniform weights, and that replacing learned weights with uniform ones does not degrade performance. SPD-MetaFormer uses uniformly weighted Fréchet aggregation with a geodesic residual and a shared spectral feedforward map, achieving competitive results on three EEG benchmarks while maintaining SPD‑valued token states until classification.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Jun 25

Towards Robust EEG Decoding Based on Riemannian Self-Attention

arXiv:2606. 25456v1 Announce Type: new Abstract: Brain-Computer Interface (BCI) based on electroencephalography (EEG) enables direct interaction between the brain and external environments and has significant applications in assistive technologies, medical rehabilitation, and entertainment.

By Shaocheng Jin, Tao Zhou, Rui Wang, Ziheng Chen, Xiaoqing Luo, Xiaojun Wu, Josef Kittler
benchmarks
More like this →
arXiv AI
Oct 5

Escaping the Capacity Ceiling: Routing on the Stiefel Manifold for Bilinear SPD Layers

The paper introduces SCAP (Stiefel Cross-Attention Pool), a routing layer that uses a pool of Stiefel filters combined via cross‑attention to create a sample‑specific bilinear map. SCAP addresses the capacity ceiling problem of standard ReEig nonlinearity in SPD networks, where a single filter cannot align multiple domains. Experiments on five cross‑domain EEG motor‑imagery datasets show SCAP improves balanced accuracy over fixed‑filter SPDNet and matches or surpasses three domain‑adaptive baselines on most datasets.

By Isabella C. Maia, Salem Said, Pedro L. C. Rodrigues, Marco Congedo
safety
More like this →
arXiv AI
Sep 10

Adaptive Anisotropic Attention for Axis-Structured Signals

The paper introduces Adaptive Anisotropic Attention (AAA), a method that splits self‑attention into temporal and spatial paths for axis‑structured signals like EEG. A learned gate combines the two paths for each token, and the resulting AXON model outperforms dense attention baselines on six EEG tasks and shows transfer to audio spectrograms. The study demonstrates that aligning attention with the natural axes of structured data provides a beneficial inductive bias.

By Mahir Jain, Parshva Runwal, Aditya Ray Mishra, Arvasu Kulkarni, Sandeep Singh, Siddharth Panwar
fine-tuningsafety
More like this →
arXiv AI
Sep 21

Rhamba: Region-Aware Hybrid Attention-Mamba Framework for Self-Supervised Learning in Resting-State fMRI

Rhamba is a region‑aware pretraining framework for resting‑state fMRI that combines anatomically guided masking with hybrid Attention‑Mamba architectures. The study pretrained models on the ABIDE dataset using three masking strategies (Any, Majority, Pure) and evaluated four architectural variants, finding that the Mamba‑Attention (MA) hybrid achieved the best average AUROC on downstream schizophrenia and ADHD classification tasks. Explainable AI via Integrated Gradients highlighted that performance depends on the interaction between masking strategy and architecture rather than a single dominant configuration.

By Pankaj Pandey, Ruthwik Reddy Doodipala, Pratheek Eranki, Carolina Torres-Rojas, Manob Jyoti Saikia, Ranganatha Sitaram
ragfine-tuningbenchmarkssafety
More like this →