arXiv AI

ENAS: An Efficient Hardware-Aware Neural Architecture Search Framework for TinyML on Resource-Constrained Microcontrollers

ENAS is a hardware‑aware neural architecture search framework tailored for TinyML on microcontrollers. It uses a static feasibility check, a cell‑based search space with various block types and skip connections, and a three‑stage hybrid search strategy (random → top‑K → mutation) with cross‑run caching. The framework runs efficiently without GPUs, achieving significant search‑time speedups and competitive accuracy on Visual Wake Words and Melanoma Cancer benchmarks across a range of microcontrollers.

arXiv Computer Vision
Nov 6

MCUFlowNet: Hardware-Aware Neural Architecture Search for Optical Flow on Microcontrollers

MCUFlowNet introduces a hardware‑aware neural architecture search framework, PACE‑NAS, that separates memory usage from latency for optical flow networks on microcontrollers. By fixing the backbone’s peak SRAM and searching for efficient operators and depths, the authors reduce peak SRAM usage by 18% and improve accuracy with lightweight context modules. The resulting MCUFlowNet models, particularly MCUFlowNet‑S, run 1.76× faster than a retrained EdgeFlowNet while achieving comparable or better performance in simulated drone flight tests, and the approach also benefits semantic segmentation tasks.

By Enmin Lin, Jona Beysens
arXiv AI
3d ago

Beyond the Trial-and-Error Loop: Hybrid Projection and Automated Tuning for Distributed Training

The paper introduces Primus Projection, a hybrid engine that combines GPU microbenchmarks with analytical scaling to predict multi‑node distributed training performance, and Primus Tuning Agent, an automated search that uses the projection engine as a fast oracle to explore the vast configuration space of parallelism and micro‑batching. Together, they reduce the need for costly trial‑and‑error on hundreds of GPUs, achieving predictions within ~10% of real throughput and enabling significant throughput gains—up to +27% in a Mixtral 8x22B case study—within minutes of single‑node exploration.

By Anshu Raina, Peyman Razaghi, Yuankai Chen, Cheng Yao, Yao Fu, Vidushi Goyal, Devang Patel, Wen Xie, Zhenyu Gu, Emad Barsoum
arXiv Machine Learning
Jul 31

SNAC-Pack 2.0: Scaled-Out Surrogate Neural Architecture Codesign

arXiv:2605. 16138v3 Announce Type: replace Abstract: Neural architecture search (NAS) is a powerful approach for automating model design, but existing methods often optimize for accuracy alone or rely on proxy metrics such as bit operations (BOPs) that correlate poorly with hardware cost.

By Jason Weitz, Dmitri Demler, Benjamin Hawks, Aaron Wang, Nhan Tran, Javier Duarte
arXiv Machine Learning
Jun 5

Surrogate Neural Architecture Codesign Package (SNAC-Pack)

arXiv:2605. 16138v2 Announce Type: replace Abstract: Neural architecture search (NAS) is a powerful approach for automating model design, but existing methods often optimize for accuracy alone or rely on proxy metrics such as bit operations (BOPs) that correlate poorly with hardware cost.

By Jason Weitz, Dmitri Demler, Benjamin Hawks, Aaron Wang, Nhan Tran, Javier Duarte
arXiv AI
Oct 31

EdgeCIM: A Hardware-Software Co-Design for CIM-Based Acceleration of Small Language Models

EdgeCIM is a hardware-software co-design framework that accelerates autoregressive decoding of small language models on edge devices. It combines a CIM macro built in 65 nm with a tile-based mapping strategy to balance pipeline stages and reduce DRAM bandwidth bottlenecks. Benchmarks show that, at INT4 precision, EdgeCIM delivers up to 7.07× higher throughput than competing edge accelerators and averages 336.4 tokens/s with 173 tokens/J across a range of models up to 4 B parameters.

By Jinane Bazzi, Mariam Rakka, Fadi Kurdahi, Mohammed E. Fouda, Ahmed Eltawil
arXiv Machine Learning
Aug 11

Gradient Under Microscope: Benchmarking Resource Utilization of Memory-Efficient Gradient Computation Methods

arXiv:2608. 08961v1 Announce Type: new Abstract: AI training's rising resource intensity is straining electricity supplies and carbon budgets, motivating systematic study of memory-efficient training on constrained hardware.

By Sarthak Mahapatra, Zihan Zhou, Khatoon Khedri, Mehdi Hosseinzadeh, Reza Rawassizadeh
arXiv AI
Jun 30

KernelSight-LM: A Kernel-Level LLM Inference Simulator

arXiv:2606. 28565v1 Announce Type: cross Abstract: As large language models (LLMs) move into production serving, practitioners must rapidly evaluate inference performance across diverse hardware, models, and serving parameters to meet cost and latency targets.

By Xiteng Yao, Taeho Kim, Hengzhi Pei, Xinle Liu, Kyle Ulrich, Leonard Lausen, Ashish Khetan, Xiang Song, George Karypis, Martin Herbordt
arXiv Machine Learning
Sep 17

Beyond Static RAG: An Adaptive, Tri-Metric Routing Framework for Efficient Long-Context Inference on Commodity GPUs

The paper introduces the Tri‑Metric Router, a deterministic, training‑free policy that chooses among Raw, Neural, and Lexical pipelines for retrieval‑augmented generation on commodity GPUs. It uses three CPU‑side signals—spatial complexity, syntactic density, and type‑token ratio—to balance VRAM headroom and latency, calibrated on LongBench qasper. The method eliminates out‑of‑memory failures and improves alignment and F1 scores compared to always‑on lexical compression without extra VRAM or training costs.

By Saipraveen Vabbilisetty, Ajay Kumar Boddepalli, Deep Narayan Mishra, Shashank Kapadia, Haoan Wang, Anupriya Sharma