BiHDTrans is a neurosymbolic binary hyperdimensional transformer that merges self‑attention with hyperdimensional computing to classify multivariate time series efficiently. It surpasses existing HD models by at least 14.47% and binary transformers by 6.67% on average, while an FPGA‑accelerated implementation reduces inference latency 39.4× compared to state‑of‑the‑art binary transformers. Even with a 64% reduction in hyperspace dimensionality, BiHDTrans remains competitive, achieving 1–2% higher accuracy with 4.4× smaller model size and nearly 50% lower latency than the full‑dimensional baseline.
arXiv:2410. 07299v3 Announce Type: replace-cross Abstract: We introduce OTIS, an open time series encoder that yields high-quality time series features for downstream deployment on any system, including resource-constrained wearables and industrial sensors.
arXiv:2606. 27449v1 Announce Type: new Abstract: Multi-head attention conventionally partitions the hidden dimension equally across all heads at every layer, enforcing an identical representational subspace dimension (dh = dmodel/h) throughout the models depth.
arXiv:2606. 12629v1 Announce Type: cross Abstract: We show that the standard basis of transformer hidden states already provides a training-free, architecture-general feature basis.
arXiv:2602.06471v2 Announce Type: replace Abstract: The architectural shape of dense Transformers has remained remarkably stable: narrow-wide-narrow feed-forward networks (FFNs) consume most non-embe...
arXiv:2606. 19538v1 Announce Type: new Abstract: Convolutional networks, recurrent networks, and transformers each encode different inductive biases -- locality, sequential memory, and content-dependent pairwise interaction -- and have remained mathematically distinct since their inception.
arXiv:2609.13947v1 Announce Type: cross Abstract: In-sensor computing reduces the cost of transmitting high-resolution image data by performing early-stage processing near the sensor. However, the lo...
arXiv:2506. 20686v2 Announce Type: replace-cross Abstract: Recent advances in biomolecular modeling have been catalyzed by models such as AlphaFold3 (AF3), which introduce science-informed changes to the transformer architecture.
arXiv:2605. 18848v3 Announce Type: replace Abstract: This paper introduces Exact Linear Attention (ELA), a mechanism that achieves linear computational complexity for Transformer attention by exploiting the exact decomposition property of kernel functions, thereby eliminating approximation error.
arXiv:2608. 03926v1 Announce Type: cross Abstract: Time series anomaly detection (TSAD) underpins applications in predictive maintenance, finance, and cloud computing, however performance remains sensitive to representation choices, especially in multivariate settings.
arXiv:2606. 07713v1 Announce Type: cross Abstract: The attention mechanism is the dominant computational bottleneck in modern transformer-based AI.
arXiv:2604. 08894v2 Announce Type: replace-cross Abstract: Spiking Neural Networks (SNNs) offer superior energy efficiency over Artificial Neural Networks (ANNs).
arXiv:2607. 15123v1 Announce Type: cross Abstract: Recent FPGAs have improved deep learning (DL) inference efficiency through dedicated tensor blocks and in-BRAM computation.