Inference efficiency

Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.

6,536 stories · RSS feed

arXiv Machine Learning
Jul 2

Leveraging Multimodality for Real-Time Classification of Transients and Variables found by the Zwicky Transient Facility

arXiv:2607. 00228v1 Announce Type: cross Abstract: Modern time-domain surveys such as the Zwicky Transient Facility (ZTF) generate hundreds of thousands of alerts each night, making real-time decisions for follow-up observations a central challenge in time-domain astronomy.

By Ved G. Shah, Nabeel Rehemtulla, Adam A. Miller, Sushant Sharma Chaudhary, Michael W. Coughlin, Antoine Le Calloch, Matthew J. Graham, Joahan Castaneda Jaimes, Theophile Jegou du Laz, Ashish A. Mahabal, Frank J. Masci, Josiah Purdum, Reed Riddle, Jesper Sollerman, Anastasia Wei, Mansi M. Kasliwal
Hugging Face Trending Papers
Jul 2

Online Segment 3D Gaussians via Launching Virtual Drones

Interactive segmentation of 3D Gaussians offers a compelling opportunity for real-time manipulation of 3D scenes, thanks to the real-time rendering capability of 3D Gaussian Splatting (3DGS). However, existing methods require a time-consuming per-scene setup - typically tens of seconds or even minutes - before interactive segmentation can begin on a raw 3DGS scene.

arXiv AI
Jul 1

Token-Sparse Medical Multimodal Reasoning via Dual-Stream Reinforcement Learning

arXiv:2606. 31599v1 Announce Type: cross Abstract: Vision-language models (VLMs) combining reinforcement learning (RL) ignite remarkable progress in multimodal reasoning, yet still struggle with medical images, which typically exhibit extremely sparse visual evidence to inform clinical decision-making.

By Kaitao Chen, Weiqian Zhao, Jiamin Wu, Qihao Zheng, Shangquan Sun, Chunfeng Song, Xiaosong Wang, Mu Zhou, Mianxin Liu
arXiv Machine Learning
Jul 1

Zero-Shot Quantization for Object Detectors using Off-the-Shelf Generative Models

arXiv:2606. 31456v1 Announce Type: new Abstract: With an increasing number of Object Detection (OD) models being deployed on edge devices, Zero-Shot Quantization for OD (ZSQ-OD) aims to quantize these models when access to the original training data is prohibited.

By Hyunho Lee, Kyomin Hwang, Hyeonjin Kim, Suyoung Kim, Sunghyun Wee, Nojun Kwak
arXiv Machine Learning
Jul 1

Sparse POD Mode Selection and Manifold Dimensionality Reduction with Neural Networks

arXiv:2605. 27756v2 Announce Type: replace-cross Abstract: Linear dimensionality reduction methods such as proper orthogonal decomposition (POD) make high-dimensional data amenable to analysis by identifying the principal components, or modes, that capture the most variance, or energy, in the data and constructing a low-dimensional representation in the subspace they span.

By Tomoki Koike, Prakash Mohan, Marc T. Henry de Frahan, Elizabeth Qian, Julie Bessac
arXiv AI
Jul 1

KV-RM: Regularizing KV-Cache Movement for Static-Graph LLM Serving

arXiv:2605. 09735v2 Announce Type: replace-cross Abstract: Static-graph LLM decoders provide predictable launches, fixed tensor shapes, and low submission overhead, but online decoding exposes highly irregular KV-cache behavior: request lengths differ, EOS events arrive asynchronously, and logical histories fragment over time.

By Zhiqing Zhong, Zhijing Ye, Jian Zhang, Weijian Zheng, Bolun Sun, Xiaodong Yu
arXiv AI
Jul 1

Real-Time Source-Free Object Detection

arXiv:2606. 31834v1 Announce Type: cross Abstract: Real-world detectors for autonomous driving, surveillance, and robotics must handle domain-shifts under strict latency and memory constraints, yet existing source-free object detection (SFOD) methods rely on heavyweight architectures that prioritize accuracy alone.

By Sairam VCR, Varun Gopal, Poornima Jain, Vineeth N Balasubramanian, Muhammad Haris Khan