arXiv:2610.07348v1 Announce Type: cross Abstract: Training large language models (LLMs) is resource-intensive, and adapting them for diverse deployment scenarios with varying computational constraint...
arXiv:2609.10018v1 Announce Type: new Abstract: EdgeAI systems are increasingly employing computer vision applications to enable intelligent, on-device decision-making in real-time. However, these de...
The paper introduces a dynamic framework for partitioning neural network layers across a heterogeneous edge‑cloud continuum, adapting to runtime changes in network conditions and device capabilities. It profiles models at startup, measures link quality, and periodically re‑evaluates the partitioning to optimize performance. Experiments on a Raspberry Pi, laptop, and desktop using VGG16, AlexNet, and MobileNetV2 demonstrate energy savings of 27.09–35.82% and latency reductions of 6.34–22.92% over static partitioning.
arXiv:2610.08268v1 Announce Type: cross Abstract: Pervasive intelligent applications are increasingly deployed on mobile and Internet of Things (IoT) edge devices. Consequently, Large Language Models...
arXiv:2610.23455v1 Announce Type: cross Abstract: LLMs and LVLMs are increasingly deployed across heterogeneous hardware, ranging from data-center GPUs to resource-constrained edge devices. Their fix...
arXiv:2606. 03770v1 Announce Type: cross Abstract: Large Language Models (LLMs) have become integral to modern applications, yet their deployment remains challenging.
arXiv:2502. 11007v5 Announce Type: replace Abstract: Compared to traditional machine learning models, recent large language models (LLMs) can exhibit multi-task-solving capabilities through multi-modal data sources and multi-turn conversations.
The paper introduces a partition-aware scheduling framework for mobile inference on heterogeneous platforms that combines mobile GPUs and multiple CPU core clusters. It jointly optimizes operator partitioning, device assignment, and execution order for static DAGs of operators, such as those in CNNs or vision transformers. An online iterative search approach decomposes large DAGs into stages, targets critical operators, and uses latency predictors to avoid exhaustive profiling, achieving near‑optimal latency with minimal scheduling overhead.
DynaFlow is a framework that enables transparent and flexible intra‑device parallelism by separating logical model definitions from physical execution schedules. It offers a flexible frontend with annotations for graph partitioning and a programmable interface for custom parallelism strategies, while its efficient backend handles asynchronous control/data‑flow, custom memory management, and remains compatible with CUDA Graphs and TorchInductor. The authors demonstrate that DynaFlow can integrate representative parallelism strategies into six state‑of‑the‑art ML systems with minimal code changes, achieving up to a 1.29× throughput improvement.
The paper presents a novel multi‑exit computational scheme for TinyML on an ultra‑low‑power GAP9 SoC, adding confidence‑based gating points to a MobileNetV2 CNN for ImageNet‑100. By allowing inference to stop early, the approach cuts average MAC operations by 41 % (from 313 MMAC to 185 MMAC), reduces inference time by 29 % (49 ms to 35 ms), and saves 24 % in energy (2.1 mJ to 1.6 mJ per frame) with only a ~1 % drop in accuracy. Compared to a state‑of‑the‑art adaptive CNN on the same hardware, the method more than doubles computational efficiency, raising MAC/cycle from 8.1 to 17.2.
The paper introduces S‑SALAAD, a framework that jointly learns model weights, low‑rank, and structured sparse components during training. It supports hardware‑compatible sparsity patterns and couples structure learning with budget‑aware compression and dedicated CPU/GPU kernels, allowing efficient inference from a single checkpoint across multiple memory budgets. Experiments on Llama2 and Qwen3 demonstrate improved perplexity, memory usage, and latency on various hardware platforms compared to baselines.
HoliBench is a modular benchmarking and deployment toolkit that jointly measures accuracy, latency, and energy for foundation models across a wide range of devices, from single-board computers to GPU servers. It provides a platform abstraction layer that calibrates cross-device measurements and supports multiple model modalities, inference engines, and quantization levels. Using HoliBench, the authors evaluated 20 models on 7 device types, revealing tradeoffs such as limited latency gains from quantization on low‑precision hardware and diminishing accuracy returns relative to energy consumption, while also showing that single-model profiles can predict multi-model pipeline performance within a few percent.