arXiv:2609.35883v1 Announce Type: cross Abstract: Genome foundation models are growing into sparse mixture-of-experts (MoE) networks whose expert weights no longer fit on the machines that hold the s...
arXiv:2601. 14033v2 Announce Type: replace Abstract: Machine learning models are increasingly served behind APIs.
SpliTEE extends the split‑inference architecture of Slalom to large language models by protecting intermediate GPU computations with differential privacy rather than encryption. The authors show that masking intermediate representations is essential, as a prompt‑reconstruction attack can recover prompts with about 80% accuracy. Their global sensitivity analysis bounds the noise needed, and they demonstrate that SpliTEE on Intel TDX achieves near‑double the speed of fully CPU‑based inference and outperforms encryption‑based Slalom while maintaining higher accuracy.
arXiv:2606. 24722v1 Announce Type: new Abstract: Frontier AI training is increasingly shaped by access to dense, centrally controlled accelerator clusters.
arXiv:2608. 02378v1 Announce Type: new Abstract: Private inference protects both user inputs and server models during neural network inference, but existing solutions remain too slow for practical deployment.
arXiv:2609.13585v1 Announce Type: cross Abstract: Communication has become a bottleneck in distributed training and inference of large models. Overlapping communication with computation at the granul...
The paper introduces Edge0, a streaming mixture‑of‑experts (MoE) inference engine that enables a 35‑billion‑parameter MoE model to run on consumer hardware by predicting routing decisions one token ahead. Edge0 uses a per‑layer prerouter to prefetch the necessary experts from SSD, and an unmerged recovery LoRA trained on the student path to recover quality lost to 4‑bit quantization and routing replacement. On a single 24‑GB machine, Edge0 serves the 35B MoE at 20 tokens per second while keeping peak active memory below 3 GiB, achieving performance close to its fp16 teacher across five public benchmarks.
arXiv:2607. 29221v1 Announce Type: cross Abstract: We address the challenge of securely and efficiently outsourcing AI computations from a trusted but computationally weak client to an untrusted but powerful server, in the setting where the client holds both the input and the model, and the server must learn neither.
arXiv:2607. 27940v1 Announce Type: new Abstract: Federated fine-tuning of large language models (LLMs) enables collaborative training without exposing raw data.
arXiv:2608. 02989v1 Announce Type: new Abstract: Speculative decoding verifies a tree of draft tokens in one target-model forward pass.
arXiv:2609.38090v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models are a compelling architecture for scaling model capacity, making them especially attractive for deployment on resource-...
arXiv:2608. 15383v1 Announce Type: new Abstract: Sparse mixture-of-experts (MoE) language models reduce arithmetic by activating only a small subset of experts per token, yet deployment still requires storing and moving the full expert bank.