arXiv Machine Learning

Tensor-Accelerated Eager Multi-Resolution Grids for Evolving Large-Scale Substrates

The paper introduces EMR‑HyperNEAT, an eager multi‑resolution grid approach that replaces the recursive quadtree subdivision of ES‑HyperNEAT with a parallelizable evaluation of all grid positions followed by a variance‑based filter. This reformulation removes sequential dependencies, enabling efficient batching across cores and population members, and reduces computational complexity from <O(4^D)> to <O(4^D/P)>. Experiments show 12–34× GPU speedups at depths 5–7 on XOR and higher solve rates across benchmarks.

arXiv AI
Sep 28

DanLing NestedTensor: Composable Multi-Ragged Tensors for Deep Learning

DanLing NestedTensor is a PyTorch tensor abstraction that embeds multi‑ragged structure directly into the tensor, allowing packed values to carry partition information and logical dimension order. This design enables broadcasting, feature transformations, and reductions to automatically respect ragged axes while preserving the same representation through autograd and both eager and compiled execution. Benchmarks on an A100 show significant speedups—up to 3.39× over padding for BERT models and 2.40–4.32× for a Pairformer‑style workload—while dramatically reducing peak memory usage.

By Zhiyuan Chen
arXiv Machine Learning
Aug 4

Nova: An End-to-End MLIR Compiler for Deep Learning

arXiv:2608. 00029v1 Announce Type: cross Abstract: The performance of deep learning models at scale relies heavily on how effectively high-level mathematical operations are mapped to underlying physical hardware.

By Adwaid Suresh, Aparna A, Harshini V M, Jona Delcy C A, Killi Uma Maheswara Rao, Ram Charan Golla, Surendra Vendra
arXiv Machine Learning
Oct 28

One Megakernel is Not Enough: Predicting Where to Cut for Streaming Inference

The paper investigates how to optimally partition streaming inference models into megakernels and conventional graph components. It introduces a framework that predicts the best megakernel composition without full end‑to‑end runs, showing that multiple megakernels or a fully conventional graph often outperform a single megakernel across various models and batch sizes. The study defines four model categories based on megakernel stability and demonstrates significant latency and throughput gains on an H100 NVL GPU using a three‑megakernel design.

By Aditya Chatterjee, Anand Joseph X M, Sanjay Singh Chauhan
arXiv Machine Learning
6d ago

Looping Deeper Without Growing the KV Cache: Recurrent Depth for Causal Encoder-Decoders

The paper introduces a looped causal encoder-decoder (CED) architecture that increases model depth without enlarging the key-value (KV) cache across layers and rounds. By placing recurrence in the encoder and allowing sparse encoder layers to generate global KV, the design achieves language‑modeling gains while maintaining long‑context scaling. Experiments show that the Looped CED matches the memory footprint of an unlooped CED and recovers most of a looped Transformer’s performance with significantly less KV, enabling higher throughput on large‑token prompts.

By Jiaqi Leng, Minshen Zhang, Zhaofeng Sun, Yu-Yang Qian, Peng Zhao, Hao Zhang
arXiv AI
Jul 22

Cost Accounting for Reactive Computational Graphs: Exhaustive Sweeps, Sequential Mutation, and the Backward-Locality Gap

arXiv:2607. 18323v1 Announce Type: cross Abstract: Exhaustive site-by-site interventions on a neural network's computational graph -- activation-patching sweeps, circuit-discovery searches, systematic ablation studies -- mutate the graph at every candidate site, and their cost is dominated by recomputation after each mutation.

By Abdallah Khemais (ISITCOM, University of Sousse)
arXiv Machine Learning
Oct 19

Peak-Memory Schedules You Can Trust: Exact Realization, Certified Optimality, and the Limits of Recomputation for Inference

The paper presents a method for reliably scheduling operator execution in real‑time inference on small devices, achieving exact peak‑memory predictions and certified optimality. By profiling the generated program and modeling its memory usage, the authors can permute statements to match the modeled peak, guaranteeing bitwise‑identical results on CPUs and characterizing GPU limitations. They also provide efficient lower‑bound traps and analyze selective recomputation, showing it is NP‑hard in general but can reduce peak memory on certain architectures.

By Ivan Fomin, Ivan Koryabkin, Boris Sheludko
arXiv Machine Learning
Oct 9

PageWeaver: KV-Guided Query Unions for Sparse Attention

PageWeaver introduces a KV‑guided query union strategy for sparse attention, assembling query groups based on selected‑page affinity while preserving each query’s support and output ownership. The design employs a bounded GPU search to generate query IDs and an ID‑aware two‑CTA kernel that consumes them without materializing reordered Q tensors or cross‑page partial outputs. Experiments on the H200 Union8 with FP8 KV show a 1.70× speedup over FlashInfer, with additional latency reductions of 3.26–7.66% on selected captures and modest throughput gains for whole‑model prefill.

By Zhiyuan Li, Zihan Li, Zefang Yuan, Lei Wang, Hao Wang