Inference efficiency

Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.

6,125 stories · RSS feed

arXiv Computation and Language
Aug 25

Training Agents to Evolve with Their Harness: TaoLive Digital Avatar Agent Technical Report

arXiv:2608.15763v2 Announce Type: replace Abstract: AI-powered digital avatar streamers must answer product questions, engage viewers, and execute marketing strategies in real time, demanding low lat...

By TaoLive AIGC LLM Team, Yuhan Sun, Wenhao Lin, Yongdong Luo, Yibo Hu, Meiguang Jin, Junfeng Ma, Weihang Pan, Jiaxin Zhao, Zulong Chen
arXiv Computer Vision
Aug 25

VeCAS: Vessel-Focused Contrast-Free Angiogram Synthesis for Vascular Interventions

VeCAS is a two‑stage framework that generates contrast‑free X‑ray angiograms from non‑contrast images. Stage I localizes vascular structures using a discriminative model and cross‑modality latent distillation, while Stage II synthesizes angiographic appearance within those regions with a vessel‑focused inpainting model. Experiments on a lower‑limb vascular intervention dataset show that VeCAS improves vascular structural fidelity and image quality, and robotic guidewire navigation tests demonstrate significant reductions in time to target and operation steps.

By De-Xing Huang, Chen-Yu Wang, Hao Liang, Xiao-Hu Zhou, Mei-Jiang Gui, Tian-Yu Xiang, Qin-Yi Zhang, Chen Wang, Xiao-Liang Xie, Shi-Qi Liu, Ming-Yuan Liu, Zhen-Chang Wang, Zeng-Guang Hou
arXiv AI
Aug 25

SlideGen: Collaborative Multimodal Agents for Scientific Slide Generation

SlideGen is a collaborative vision‑language multi‑agent framework designed to generate scientific presentation slides from research papers. It assigns specialized agents to outline the presentation structure, align figures and tables with key claims, generate speaker notes, and compose editable PPTX slides using a diverse layout library. The system introduces a geometry‑aware density metric to evaluate visual clutter and demonstrates significant improvements in layout balance, content coverage, and text coherence over existing baselines on a 200‑paper benchmark.

By Xin Liang, Zhilin Zhang, Xiang Zhang, Haoran Su, Yiwei Xu, Siqi Sun, Chenyu You
arXiv AI
Aug 25

EG-ARSA: An Expert-Grounded Open Model for Visual Road Safety Auditing in Low-Resource Settings

EG-ARSA introduces an Expert‑Grounded Distillation (EGD) framework that transfers institutional road safety expertise into a compact vision‑language model for visual road safety auditing. The method calibrates a teacher model against authoritative field audits, achieving a Cohen’s kappa of 0.74 before generating structured supervision for an 8‑billion‑parameter student model via Low‑Rank Adaptation. The authors also release Bangladesh Road Safety Audit (BD‑ARSA), an open dataset of 21,947 image‑audit records, and demonstrate that the student model outperforms both its larger teacher and Gemini‑2.5‑Flash in ordinal risk assessment and expert evaluation.

By Md Thamed Bin Zaman Chowdhury, Moazzem Hossain
arXiv Computer Vision
Aug 25

SemanticXR: Low Power and Real-time Queryable Semantic Mapping with an Object-Level Device-Cloud Architecture

SemanticXR is a device‑cloud system that enables real‑time, open‑vocabulary semantic mapping and querying for XR applications while respecting power, bandwidth, and memory limits. By treating semantically identifiable objects as first‑class units, the system coordinates communication, execution, and memory across device and server, achieving a 2.2× faster server‑side mapping latency and keeping upstream bandwidth below 2.5 Mbps. On the device, an object‑level sparse local map with incremental updates delivers sub‑100 ms query latency for up to 10,000 objects, supports tens of thousands of objects within a 500 MB footprint, and adds only about 2 % to idle power.

By Rahul Singh, Devdeep Ray, Connor Smith, Sarita Adve
arXiv AI
Aug 25

From Isolation to Alignment: Unified LoRA for Efficient Multi-Task Learning

The paper introduces Align‑LoRA, a unified LoRA framework for multi‑task learning that replaces complex, isolated adapter designs with a single‑adapter model enhanced by a higher rank and an explicit alignment loss. It demonstrates that a router‑free, multi‑head model with high inter‑head redundancy can outperform more elaborate baselines, and that a unified LoRA can achieve competitive performance while enabling weight merging and zero inference latency. Extensive experiments and theoretical analysis confirm that Align‑LoRA surpasses prevailing approaches, offering a simpler, production‑friendly paradigm for parameter‑efficient fine‑tuning of large language models.

By Jinda Liu, Yi Chang, Yuan Wu
arXiv Computer Vision
Aug 25

Mapping the Concept Landscape: Structural Perception of Global Distributions for Transparent Data Pruning

The paper introduces Mapping the Concept Landscape (MCL), a framework that replaces high‑dimensional feature embeddings with explicit sample‑level graphs of entities, events, and attributes for image‑caption pairs. By aggregating these graphs into a dataset‑level graph, MCL captures the global distribution of semantic concepts and identifies rare concepts. A greedy algorithm then selects samples to maximize coverage of under‑represented concepts, achieving better pruning efficiency and providing a transparent audit trail.

By Dongyue Wu, Tao Ma