Inference efficiency

Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.

6,125 stories · RSS feed

arXiv Machine Learning
Aug 18

A Unified Geometric Framework for Developmental Analysis of Spatial Transcriptomic Data

arXiv:2608. 15306v1 Announce Type: cross Abstract: High-throughput single-cell and spatial transcriptomic technologies provide high-resolution snapshots of heterogeneous cellular states, but their destructive nature prevents repeated measurements of the same cells over time.

By Mary Chriselda Antony Oliver, Kaitlyn Hohmeier, Tuyen Tran, Alejandra Castillo, Caroline Moosm\"uller, Shiying Li
arXiv Machine Learning
Aug 18

CodeQuant: Unified Clustering and Quantization for Enhanced Outlier Smoothing in Low-Precision Mixture-of-Experts

arXiv:2604. 10496v2 Announce Type: replace Abstract: Outliers have emerged as a fundamental bottleneck in preserving accuracy for low-precision large models, particularly within Mixture-of-Experts (MoE) architectures that are increasingly central to large-scale language modeling.

By Xiangyang Yin, Xingyu Liu, Tianhua Xia, Bo Bao, Vithursan Thangarasa, Valavan Manohararajah, Eric Sather, Sai Qian Zhang
arXiv AI
Aug 18

ACTS-SQL: Agentic and Critic-Oriented Tree-Structured SQL Correctness with Large Language Models

arXiv:2608. 15145v1 Announce Type: new Abstract: Large Language Models (LLMs) have been increasingly adopted in Text-to-SQL systems, yet SQL errors remain a major obstacle in real-world Text-to-SQL inference pipelines.

By Xinmei Huang, Jie Song, Peng Li, Fuxin Jiang, Jing Zhang, Tieying Zhang, Jianjun Chen, Chenming Liu, Tao Yang, Maoyin Liu, Wenda Li, Hong Chen, Cuiping Li
arXiv AI
Aug 18

MINT: Min-Selection Preference Distillation for Balanced Multi-Objective Alignment

arXiv:2608. 14828v1 Announce Type: new Abstract: Aligning a language agent to several objectives at once is a persistent failure mode of preference-based training: when objectives are combined additively, optimization collapses onto whichever is cheapest to improve and sacrifices the rest, so a support agent learns to sound warm while giving no real help.

By Tony Tu, Sayan Chakraborty, Ruomeng Xu, Tony Qin, Austin Tian
arXiv AI
Aug 18

Privacy-Preserving Decentralized Federated Learning via Explainable Adaptive Differential Privacy

arXiv:2509. 10691v3 Announce Type: replace-cross Abstract: Decentralized federated learning enables collaborative model training without a central server, but shared model updates can still leak sensitive information through inversion, reconstruction, and membership inference attacks.

By Fardin Jalil Piran, Zhiling Chen, Yang Zhang, Qianyu Zhou, Jiong Tang, Farhad Imani
arXiv Machine Learning
Aug 18

A Privacy Study of Sparse Collaborative Inference

arXiv:2608. 16236v1 Announce Type: new Abstract: Collaborative inference (CI) splits a model between an edge device and a server, whereby the client computes an intermediate activation, transmits it, and the server completes the computation.

By Maximilian Andreas Hoefler, Karsten Mueller, Wojciech Samek
arXiv AI
Aug 18

OceanDepths: A Global Dataset of Paired Subsurface and Surface Ocean Observations

OceanDepths is the first open, global, AI‑ready dataset that pairs satellite‑derived sea surface temperature, salinity, and height with co‑located EN4 subsurface temperature and salinity profiles, complemented by GLORYS12 reanalysis data. It covers 2000–2024 at 0.1°×0.1° spatial resolution and weekly temporal resolution, providing over 9.5 million paired profiles interpolated to 50 depth levels. The dataset’s 4‑D multivariate structure, high resolution, long temporal extent, and extreme sparsity of subsurface observations make it a challenging testbed for novel AI methods, with demonstrated use in subsurface state reconstruction and potential for observation‑based forecasting.

By Simon Donike, Ruben Cartuyvels, Antonino Ian Ferola, Elisa Carli, Diego Fernandez Prieto, Marie-Helene Rio