← Back to all news
arXiv AI By Jiawei Li, Fabio Bonassi, Johan Sundstr\"om, Thomas B. Sch\"on, Ant\^onio H. Ribeiro

On the role of the tokenizer in ECG transformer models

Read the original on arXiv AI →

The study investigates how different tokenization methods affect ECG Transformer models by comparing eight strategies across four backbone architectures on the CPSC2018 classification task. Physiology-aware tokenizations such as Median-beat and HeartLang achieve higher mean macro-AUCs (0.893 and 0.889) than point-wise and patch-wise approaches (0.822 and 0.824), while also reducing sequence length and training memory usage. Combining the two physiology-aware representations further improves macro-AUC by 8.2%.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Sep 1

Beat-Synchronous Tokenization for ECG Transformers

arXiv:2608.30367v1 Announce Type: new Abstract: Transformer-based electrocardiogram (ECG) models commonly tokenize waveforms into fixed temporal patches. Though convenient, fixed patching can split h...

By Ahmed Sameh, Nolan Wilson, Max Enderlein, Yogatheesan Varatharajah
llmsnlp
More like this →
arXiv AI
Sep 21

BEAT-Net: Injecting Biomimetic Spatio-Temporal Priors for Interpretable ECG Diagnosis

BEAT-Net is a supervised biomimetic framework for ECG diagnosis that incorporates QRS-centered tokenization and a hierarchical architecture mirroring a cardiologist’s workflow. It processes heartbeat sequences through morphological, spatial, temporal, and transformer-based stages, achieving an AUC of 0.924 on large benchmarks while using only 0.7 million parameters. The model outperforms the 39.5‑million‑parameter HeartLang foundation model on morphological form classification and demonstrates superior cross‑dataset generalization with only 35% of the training data.

By Runze Ma, Haonan Lyu, Shunbo Jia, Qiang Yang, Muzi Xu, Jiaqi Zhang, Zihe Luo, Caizhi Liao
llmsnlpbenchmarks
More like this →
arXiv Computer Vision
Sep 21

ECG-Mamba-V2: Architectural Refinements to a Bidirectional State Space Model for Multi-Label 12-Lead ECG Classification

ECG-Mamba-V2 refines a bidirectional Vision Mamba encoder for multi‑label 12‑lead ECG classification by appending the class token at the sequence end, summing forward and backward outputs without ½ scaling, and applying uniform dropout across blocks. These architectural tweaks yield a 0.6494 macro AUPRC and 0.9716 macro AUROC on the PhysioNet/CinC Challenge 2021, outperforming its predecessor while using 34% fewer parameters and achieving 38% higher throughput.

By Huawei Jiang, Husna Mutahira, Shibo Wei, Gan Huang, Vladimir Shin, Dongryeol Ryu, Juneho Yi, Mannan Saeed Muhammad
efficiency
More like this →
arXiv Computer Vision
Sep 25

A Hybrid CNN--State-Space--Attention Backbone with Joint-Embedding Predictive Pretraining for 12-Lead ECG Classification

The paper presents a hybrid CNN–state‑space–attention backbone designed for 12‑lead ECG classification, combining early waveform tokenization, mixed temporal dynamics modeling, and late global attention. It introduces an ECG‑oriented Joint‑Embedding Predictive Pretraining (JEPA) that samples span masks at latent resolution and predicts clean latent targets via a momentum encoder, avoiding waveform reconstruction. Experiments on CPSC2018, Chapman‑Shaoxing, and PTB‑XL, with pretraining on ~350K unlabeled CODE‑15 recordings, demonstrate strong supervised baselines and improved transfer, especially in low‑label scenarios and with LoRA adaptation.

By Yakoub Bazi, Sarah Aljuhani, Mohamad M. Al Rahhal, Mansour Zuair, Naif Alajlan
ragnlpfine-tuning
More like this →