arXiv Machine Learning June 16, 2026 By Augusto Peres, Iker Perez, Pedro Valdeira, Guilherme Jardim, Ana Sofia Gomes, Hugo Ferreira, Pedro Bizarro

Decoupling Inference from State Updates in Low-Latency Feature Engines via Probabilistic Thinning

Read the original on arXiv Machine Learning →

arXiv:2606. 16981v1 Announce Type: cross Abstract: Streaming data systems increasingly underpin Machine Learning workflows that maintain large numbers of continuously updated aggregations.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Aug 3

OnlineCache: Learning Dynamic Caching Policies with Error Correction for Efficient Diffusion Inference

arXiv:2607. 29398v1 Announce Type: new Abstract: Diffusion models have revolutionized generative tasks but incur high latency due to iterative denoising.

By Zhikang Xie, Xichen Ye, Yifan Wu, Haoshen Yu, Li chenan, Peizhu Gong, Weizhong Zhang, Cheng Jin

diffusion reinforcement-learning

arXiv Machine Learning

Jul 7

AdaptiveSD A Stability-Aware, Runtime-Adaptive Speculative Decoding Framework with Multi-Policy Orchestration for CPU-Constrained LLM Inference

arXiv:2607. 03876v1 Announce Type: new Abstract: With the rise of small quantized GGUF-based language models and their increasing use for on-device inference tasks, we have seen the growing need for an approach capable of reliably delivering these models at scale even under severe memory bandwidth constraints such as those imposed by pure CPU implementations.

By Sadra Saremi

llms agents reinforcement-learning efficiency

arXiv AI

Jul 28

Eviction as Estimation: A Fixed-Lag Smoothing View of Test-Time Memory, and When Measuring Beats Accumulating

arXiv:2607. 24667v1 Announce Type: new Abstract: A language model with a bounded working memory must repeatedly decide which stored items to keep.

By Maruthi Vemula, Neeraj Praneeth Gajula

llms nlp benchmarks

arXiv Machine Learning

Jun 19

Execution-State Capsules: Graph-Bound Execution-State Checkpoint and Restore for Low-Latency, Small-Batch, On-Device Physical-AI Serving

arXiv:2606. 20537v1 Announce Type: new Abstract: Mainstream LLM serving systems reuse prefix work mainly through paged or radix key-value (KV) caches.

By Liang Su

llms agents robotics efficiency benchmarks data-engineering

arXiv Machine Learning

Jun 18

Beyond Prediction: Tail-Aware Scheduling for LLM Inference

arXiv:2606. 18431v1 Announce Type: new Abstract: LLM serving exhibits extreme length variability, making size-based scheduling difficult in practice.

By Yueying Li, Yuanfan Chen, Jiayang Chen, Esha Choukse, Haoran Qiu, G. Edward Suh, Rodrigo Fonseca, Ziv Scully, Udit Gupta

llms

arXiv Machine Learning

Jul 31

Back from the Future: Key-Value Cache Management by Counter-Causal Surprise

arXiv:2607. 27600v1 Announce Type: new Abstract: Key-value (KV) cache management through compression and eviction strategies has emerged as an important research direction in recent years.

By Stephen Gould, Anton van den Hengel

llms efficiency multimodal benchmarks

Related stories

OnlineCache: Learning Dynamic Caching Policies with Error Correction for Efficient Diffusion Inference

AdaptiveSD A Stability-Aware, Runtime-Adaptive Speculative Decoding Framework with Multi-Policy Orchestration for CPU-Constrained LLM Inference

Eviction as Estimation: A Fixed-Lag Smoothing View of Test-Time Memory, and When Measuring Beats Accumulating

Execution-State Capsules: Graph-Bound Execution-State Checkpoint and Restore for Low-Latency, Small-Batch, On-Device Physical-AI Serving

Beyond Prediction: Tail-Aware Scheduling for LLM Inference

Back from the Future: Key-Value Cache Management by Counter-Causal Surprise