arXiv Machine Learning September 15, 2026 By Zhuojin Li, Marco Paolieri, Leana Golubchik

Partition-Aware Scheduling for Mobile Heterogeneous Inference Co-Execution

Read the original on arXiv Machine Learning →

The paper introduces a partition-aware scheduling framework for mobile inference on heterogeneous platforms that combines mobile GPUs and multiple CPU core clusters. It jointly optimizes operator partitioning, device assignment, and execution order for static DAGs of operators, such as those in CNNs or vision transformers. An online iterative search approach decomposes large DAGs into stages, targets critical operators, and uses latency predictors to avoid exhaustive profiling, achieving near‑optimal latency with minimal scheduling overhead.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

1d ago

DynaFlow: Transparent and Flexible Intra-Device Parallelism via Programmable Operator Scheduling

arXiv:2605.21603v1 Announce Type: cross Abstract: Intra-device parallelism addresses resource under-utilization in ML inference and training by overlapping the execution of operators with different r...

By Yi Pan, Yile Gu, Jinbin Luo, Yibo Wu, Ziren Wang, Hongtao Zhang, Ziyi Xu, Shengkai Lin, Baris Kasikci, Stephanie Wang

efficiency benchmarks

arXiv AI

Jul 14

Automated Tensor Scheduling for Hybrid CPU-GPU LLM Inference on Consumer Devices

arXiv:2607. 10183v1 Announce Type: cross Abstract: Running large language models on consumer devices such as laptops and desktops is challenging because model weights often exceed GPU memory capacity, making offloading inference necessary to extend effective model capacity with CPU memory.

By Yangyijian Liu, Hongyi Ye, Mingyang Li, Wu-jun Li

llms efficiency

arXiv Machine Learning

Sep 14

HoliBench: A Cross-Platform Benchmarking and Deployment Toolkit for Foundation Models in CPS-IoT Applications

HoliBench is a modular benchmarking and deployment toolkit that jointly measures accuracy, latency, and energy for foundation models across a wide range of devices, from single-board computers to GPU servers. It provides a platform abstraction layer that calibrates cross-device measurements and supports multiple model modalities, inference engines, and quantization levels. Using HoliBench, the authors evaluated 20 models on 7 device types, revealing tradeoffs such as limited latency gains from quantization on low‑precision hardware and diminishing accuracy returns relative to energy consumption, while also showing that single-model profiles can predict multi-model pipeline performance within a few percent.

By Inesh Chakrabarti, Zejun Xiong, Pragya Sharma, Mani Srivastava

llms efficiency multimodal benchmarks

arXiv Machine Learning

Aug 24

DAOP: Data-Aware Offloading and Predictive Pre-Calculation for Efficient MoE Inference

arXiv:2501.10375v3 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) models, though highly effective for various machine learning tasks, face significant deployment challenges on memory...

By Yujie Zhang, Shivam Aggarwal, Tulika Mitra

arXiv Machine Learning

3d ago

Mira: Memory-Efficient MoE Inference Using Adaptive Caching and Predictive Expert Staging

arXiv:2609.38090v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models are a compelling architecture for scaling model capacity, making them especially attractive for deployment on resource-...

By Sanjali Yadav, Bahar Asgari

efficiency benchmarks

arXiv AI

Jul 21

A Hardware-oriented Approach for Efficient Bayesian Inference Computation and Deployment

arXiv:2607. 17855v1 Announce Type: new Abstract: Bayesian inference provides a principled foundation for reasoning under uncertainty, but its computational cost hinders deployment on resource-constrained edge devices.

By Nikola Pi\v{z}urica, Matteo Risso, Nikola Milovi\'c, Alessio Burrello, Igor Jovan\v{c}evi\'c, Conor Heins, Miguel de Prado

reinforcement-learning efficiency

One email a morning, machine-written

Related stories

DynaFlow: Transparent and Flexible Intra-Device Parallelism via Programmable Operator Scheduling

Automated Tensor Scheduling for Hybrid CPU-GPU LLM Inference on Consumer Devices

HoliBench: A Cross-Platform Benchmarking and Deployment Toolkit for Foundation Models in CPS-IoT Applications

DAOP: Data-Aware Offloading and Predictive Pre-Calculation for Efficient MoE Inference

Mira: Memory-Efficient MoE Inference Using Adaptive Caching and Predictive Expert Staging

A Hardware-oriented Approach for Efficient Bayesian Inference Computation and Deployment