The study investigates how different tokenization methods affect ECG Transformer models by comparing eight strategies across four backbone architectures on the CPSC2018 classification task. Physiology-aware tokenizations such as Median-beat and HeartLang achieve higher mean macro-AUCs (0.893 and 0.889) than point-wise and patch-wise approaches (0.822 and 0.824), while also reducing sequence length and training memory usage. Combining the two physiology-aware representations further improves macro-AUC by 8.2%.
By Jiawei Li, Fabio Bonassi, Johan Sundstr\"om, Thomas B. Sch\"on, Ant\^onio H. Ribeiro
The paper presents a hybrid CNN–state‑space–attention backbone designed for 12‑lead ECG classification, combining early waveform tokenization, mixed temporal dynamics modeling, and late global attention. It introduces an ECG‑oriented Joint‑Embedding Predictive Pretraining (JEPA) that samples span masks at latent resolution and predicts clean latent targets via a momentum encoder, avoiding waveform reconstruction. Experiments on CPSC2018, Chapman‑Shaoxing, and PTB‑XL, with pretraining on ~350K unlabeled CODE‑15 recordings, demonstrate strong supervised baselines and improved transfer, especially in low‑label scenarios and with LoRA adaptation.
By Yakoub Bazi, Sarah Aljuhani, Mohamad M. Al Rahhal, Mansour Zuair, Naif Alajlan
arXiv:2603. 19100v2 Announce Type: replace Abstract: Electroencephalography (EEG) enables non-invasive monitoring of brain activity across clinical and neurotechnology applications, yet building foundation models for EEG remains challenging due to differing electrode topologies and computational scalability, as Transformer architectures incur quadratic sequence complexity.
By Dana\'e Broustail, Anna Tegon, Thorir Mar Ingolfsson, Yawei Li, Luca Benini
RobECG-CL is a rank‑aware contrastive learning framework designed to learn robust representations for paper ECG images, which often suffer from varied layouts, physical artifacts, and limited labels. The method constructs progressively degraded views from standard 12‑lead ECG recordings and trains the model to maintain invariance to the same recording while respecting degradation ordering. In synthetic stress tests on CODE‑II and EchoNext, RobECG‑CL shows improved robustness under severe degradation and few‑shot transfer, outperforming other contrastive baselines and surpassing the waveform‑based foundation model ECG‑FM in a 1% labeled setting; it also achieves the best macro AUROC on 312 hospital samples with 37 labels.
By Yinghao Xie, Zhenbang Dai, Haojun Wang, Jinyu Cai, Fabio Bonassi, Hongwu Chen, Johan Sundstr\"om, Jiawei Li, Ant\^onio H. Ribeiro
arXiv:2608.30367v1 Announce Type: new
Abstract: Transformer-based electrocardiogram (ECG) models commonly tokenize waveforms into fixed temporal patches. Though convenient, fixed patching can split h...
By Ahmed Sameh, Nolan Wilson, Max Enderlein, Yogatheesan Varatharajah
arXiv:2609. 15498v1 Announce Type: new Abstract: Recurrent Transformers reusing their weights rather than stacking $L$ distinct layers are becoming widely adopted due to their parameter efficiency [1,2,3].
By Pawel Olszowiec, Michal Byra, Grzegorz Gruszczynski, Grzegorz Stefanski, Alberto Presta