Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.
We study the problem of demonstration selection, which involves selecting a subset of examples for prepending to a query to a language model. This problem is closely related to in-context learning and...
Neural audio codecs are a key component in speech language modeling. However, their high frame rates lead to long sequence lengths, increasing computational costs. Dynamic frame rate codecs mitigate t...
This thesis develops robust and efficient AI frameworks for accelerating crystalline materials discovery by addressing both major stages of the materials-design pipeline: crystal property prediction a...
A common small-model deployment runs one shared backbone with several LoRA specialists that answer over the same context. Serving them naively re-prefills that shared context once per specialist. We s...
On-policy distillation (OPD) has emerged as an effective approach for large language model post-training, yet existing objectives face a trade-off between objective fidelity and optimization stability...
Large-scale vision-language models (VLM) such as CLIP enable strong open-vocabulary reasoning, yet deploying these capabilities on resource-constrained edge devices remains challenging. EdgeVL address...
State space models (SSMs), particularly Mamba, have emerged as efficient alternatives to attention-based architectures and have been extended to vision through ViM, VMamba, and Visual State Space Dual...
MorphoStyle is a new framework for shape‑aware motion style transfer that uses a shape‑conditioned FSQ‑VAE. It disentangles style from content through a contrastive style encoder, a text‑guided style‑routing mechanism, and a manifold‑preserving style modulator. Experiments on benchmark datasets show that MorphoStyle outperforms existing baselines in both shape control and motion style transfer.
arXiv:2609.13285v1 Announce Type: cross Abstract: The KV cache is a primary bottleneck for Transformer decoding: its memory footprint and cache-read traffic grow with sequence length. Grouped-query a...
MAPS is a Memory-Aware Predictive Scheduling framework designed for disaggregated large language model (LLM) serving. It uses device-assisted speculative output length prediction and uncertainty-aware calibration to establish safe output-length upper bounds, which inform a hierarchical global-local scheduling strategy that reduces queue buildup and head-of-line blocking. Experiments on real-world workloads and two LLMs demonstrate that MAPS lowers average end-to-end latency by 42.6% and tail latency by up to 84.8% compared to three state-of-the-art systems.
The paper introduces PriCoder, a method for teaching large language models (LLMs) to effectively use private library APIs for code generation. PriCoder synthesizes training data by constructing a graph and applying two operators—Progressive Graph Evolution to increase diversity and Multidimensional Graph Pruning to enhance quality. Experiments on three mainstream LLMs demonstrate that PriCoder boosts private‑library code generation by over 20% in pass@1, while leaving general code generation largely unchanged.
arXiv:2609.13199v1 Announce Type: new Abstract: Knowledge distillation aims to improve the performance of lightweight student models by transferring knowledge from larger and more powerful teacher mo...
MANE is a distributed inference framework that uses a multi‑path tail architecture to allow dynamic accuracy–throughput trade‑offs during edge onloading of deep neural networks. It introduces a novel multi‑path model, a three‑stage training scheme with Joint Head Network Distillation loss, and a hysteresis‑based scheduler with an equitable device‑fallback policy. The system achieves over 80% SLO satisfaction and 6pp higher accuracy than on‑device alternatives while supporting up to 40 concurrent devices.
The paper investigates how much learned memory is required to leverage additional data in autoregressive prediction models. It introduces a predictive‑energy spectrum that jointly governs data and memory scaling, proving a minimax law that links the number of prediction blocks and the size of the learned state to this spectrum. The authors demonstrate that optimal bit allocation and masked query‑key attention mechanisms realize this law, and they provide experimental evidence across multiple pretrained‑model scales.
arXiv:2609.13592v1 Announce Type: cross Abstract: GPU memory bandwidth and capacity limit throughput in large language model (LLM) inference. The GPU memory system consists of a primary tier of high-...
The paper discusses tensorizing neural networks by reshaping dense weight matrices into higher-order tensors and approximating them with low-rank tensor network decompositions. This approach offers promising model compression and introduces bond indices that create new latent spaces, potentially enhancing interpretability. Despite encouraging empirical results, tensorized neural networks remain underused, and the authors call for more research to address practical scaling and adoption challenges.
arXiv:2609.13271v1 Announce Type: cross Abstract: Medical image segmentation needs diverse training data, but hospitals hold complementary scans they cannot share for privacy and regulatory reasons....
The paper compares Complement Naive Bayes (NB) with zero‑shot and few‑shot large language models (LLMs) across a wide range of model sizes and text classification tasks. NB outperforms LLMs when labeled data is available, achieving comparable accuracy to large LLMs while running thousands of samples per second on a CPU. In zero‑data sentiment settings, LLMs still dominate, but NB remains the best choice for resource‑constrained HPC practitioners, and the authors provide a Kubernetes Helm operator to automate model selection.
arXiv:2609.15972v1 Announce Type: cross Abstract: As language models become more capable, long-term collaboration in learning, reasoning, and decision-making calls for a deeper understanding of the p...
The paper introduces a modular correction framework for large language models that uses Activated LoRA adapters and a context-aware routing mechanism to mitigate harmful outputs. By allowing expert adapters to activate mid-sequence without invalidating the KV cache, the system achieves low-latency, targeted correction during generation. Experiments show improved alignment on safety benchmarks while maintaining task performance, presenting a lightweight, scalable approach to safer LLM deployments.