arXiv Machine Learning
Seeing at the Edge: Energy Characterization of Vision-Language Model Inference on Edge Platforms
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jul 13
Seeing is Free, Speaking is Not: Uncovering the True Energy Bottleneck in Edge VLM Inference
arXiv:2607. 09520v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) are the perceptual backbone of embodied AI, but their energy footprint on edge hardware remains poorly understood.
arXiv Machine Learning
4d ago
Shared Context, Separate Engines: Streaming Vision--Language Inference on Edge SoCs
The paper introduces a system that coordinates heterogeneous accelerators on edge SoCs to run vision–language model inference concurrently, separating a stream’s context from each engine’s key‑value state. By allowing engines to import, reuse, or reconstruct state, the method supports stateful streaming generation and concurrent sequence generation, achieving higher throughput and lower latency than single‑engine baselines. Experiments on Apple M4 Max, Rockchip RK3588, and Qualcomm QCS6490 show significant gains, including up to 27.2% more throughput and 72.7% faster query completion.
arXiv AI
5d ago
VLA-HWBench: Where the Time Goes When VLA Models Run on Edge Hardware
arXiv:2610.34717v1 Announce Type: cross Abstract: Vision--language--action (VLA) models increasingly run on edge devices aboard robots, where inference latency constrains closed-loop robot control. H...
arXiv AI
Sep 28
Energy Efficiency of Locally Deployed LLMs: A Preliminary Quantitative GPU Power Benchmark on Consumer Hardware
The paper reports a reproducible GPU power benchmark for 18 open‑source LLMs (0.5B–7B parameters) run on a single consumer RTX 4060ti GPU using the Ollama inference engine. Energy metrics such as mean/peak power, total energy per prompt, energy per output token, and throughput were measured, revealing that model architecture and quantization strategy, rather than parameter count alone, drive energy efficiency. The most efficient models were qwen2.5:0.5b and tinyllama:1.1b, while the 7B‑Mistral model consumed up to 8.6× more energy per token, and qwen3.5:0.8b(on) showed unusually high per‑prompt energy due to extended internal reasoning.
arXiv Machine Learning
Jul 10
Rethinking Small VLM Quantization: From Component-Wise Analysis to Hardware-Aware Edge Deployment
arXiv:2607. 08029v1 Announce Type: new Abstract: The emergence of vision language models with fewer than 3 billion parameters has accelerated the implementation of on-device multimodal intelligence.
arXiv AI
Sep 10
A Measurement Study of LLM Inference Trade-offs Across Edge Continuum Hardware
arXiv:2609.08307v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as backends for intelligent web services, but serving them across the edge continuum requires bala...