arXiv:2607. 03652v1 Announce Type: cross Abstract: Transformer blocks are prevalent in large language model (LLM) but present deployment challenges due to their challenging computational and memory demands.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
The Transformer Accelerator (TFA) is a synthesizable, parameterizable INT8 memory‑to‑memory engine designed for transformer inference and machine translation. It features a one‑time‑multiplexed datapath that handles prompt processing and autoregressive generation, and implements key operations such as matrix multiplication, softmax, RMSNorm, and elementwise functions through eight 512‑bit macro‑op descriptors. In extensive verification, TFA achieved zero mismatches across 25 tests and 34 constrained‑random runs, matched floating‑point references on multiple translation tasks, and delivered a 20× speedup over a 22‑thread CPU while projecting significant energy reductions in larger designs.
HBQ: Hierarchical Scaling Block Quantization with Hardware‑Efficiency‑Aware Design for Accurate LLM Inference proposes a new block‑quantization scheme that uses large blocks and low‑overhead significand scaling to balance hardware efficiency and accuracy. The authors demonstrate that larger blocks improve efficiency by amortizing dequantization and accumulation costs, while their SIG scaling compensates for the resulting accuracy loss. Experiments on a 28 nm ASIC accelerator show that HBQ achieves up to 4.6× higher area/energy efficiency than state‑of‑the‑art weight‑only quantization, with 1.5–3.0× speedup and 1.6–3.3× system energy reduction over existing BQ methods.
arXiv:2606. 31938v1 Announce Type: cross Abstract: Deploying Vision Transformer (ViT) models on edge platforms remains challenging due to their high computational demands and the architectural heterogeneity of modern hybrid ViT models, which incorporate both fully connected and convolutional layers.
arXiv:2605. 26092v4 Announce Type: replace-cross Abstract: The deployment of Large Language Models (LLMs) and Vision Transformers (ViTs) on edge devices is significantly constrained by memory limitations and the critical timing bottlenecks introduced by dense Multiply-Accumulate (MAC) arrays.
FlashAttention-V is a blocked FlashAttention implementation optimized for scalable vector architectures, designed to reduce the memory bandwidth bottleneck of transformer attention on CPUs. By fusing operations, exploiting parallelism across attention heads, and inter‑head packing, it improves vector register utilization and memory locality, enabling efficient scaling from short to very long vectors. Benchmarks on TinyLlama, Llama 3.2, Qwen2.5, and Pythia‑410M show 22×–42× speedups over scalar FlashAttention in prefill and 8×–11× in decode on a Banana Pi BPI‑F3, while also revealing quantization‑related bottlenecks that limit long‑vector scalability.
This paper presents an energy-efficient hardware acceleration of the convolutional layers in the U-Net architecture for image segmentation, implemented on FPGA. While digit-serial arithmetic, particularly most-significant-digit-first (MSDF) techniques, offers a compact hardware footprint, it suffers from initial latency before producing the first output digit.