arXiv:2608. 00029v1 Announce Type: cross Abstract: The performance of deep learning models at scale relies heavily on how effectively high-level mathematical operations are mapped to underlying physical hardware.
By Adwaid Suresh, Aparna A, Harshini V M, Jona Delcy C A, Killi Uma Maheswara Rao, Ram Charan Golla, Surendra Vendra
arXiv:2607. 25504v1 Announce Type: cross Abstract: Fine-grained weight pruning and activation sparsification have emerged as effective approaches for reducing the compute and memory cost of inference for Transformer models.
By Bowen Wang, Chi Zhang, Diyou Shen, Renzo Andri, Navaneeth Kunhi Purayil, Luca Benini
arXiv:2609.26147v1 Announce Type: new
Abstract: Modern LLMs are deployed as families of post-trained variants (base, instruct, chat, code) derived from a shared set of pre-trained weights. We present...
By Zhaohui Wang
The paper introduces GaugeLasso, a method that applies symmetric group‑lasso penalties to transformer channels during training, enabling entire tensor slices to be zeroed out while maintaining dense tensors for GPU efficiency. By calibrating channel penalties based on inference utility per compute, the network self‑organizes into depth‑dependent structural profiles that can be dramatically smaller than the original architecture, achieving up to 255‑fold compression on a polynomial division task and outperforming hand‑designed baselines on language modeling and autoencoding benchmarks. The approach also accelerates training and reveals over‑provisioned axes that guide subsequent design iterations.
By Jed A. Duersch, Na\"im Es-Sebbani, Nathana\"el Haas, Zied Bouraoui
arXiv:2607.12550v3 Announce Type: replace-cross
Abstract: The key-value (KV) cache has become the dominant memory cost of transformer inference: it grows with batch size, context length, and depth, a...
By Rahul Krishnan, Volker Schulz
The paper introduces EMR‑HyperNEAT, an eager multi‑resolution grid approach that replaces the recursive quadtree subdivision of ES‑HyperNEAT with a parallelizable evaluation of all grid positions followed by a variance‑based filter. This reformulation removes sequential dependencies, enabling efficient batching across cores and population members, and reduces computational complexity from <O(4^D)> to <O(4^D/P)>. Experiments show 12–34× GPU speedups at depths 5–7 on XOR and higher solve rates across benchmarks.
By Romain Claret, Michael O'Neill, Paul Cotofrei, Kilian Stoffel