SpliTEE extends the split‑inference architecture of Slalom to large language models by protecting intermediate GPU computations with differential privacy rather than encryption. The authors show that masking intermediate representations is essential, as a prompt‑reconstruction attack can recover prompts with about 80% accuracy. Their global sensitivity analysis bounds the noise needed, and they demonstrate that SpliTEE on Intel TDX achieves near‑double the speed of fully CPU‑based inference and outperforms encryption‑based Slalom while maintaining higher accuracy.
arXiv:2609.15039v3 Announce Type: replace-cross Abstract: User prompts provided to large language models (LLMs) may contain private information. One way to protect them is to execute the LLM inside a...
Open-source large language models (LLMs) are increasingly competitive with closed-source models while offering transparency and the ability to run inference without exposing user inputs to a service p...
TensorCommitments (TCs) is a lightweight, tensor-native proof‑of‑inference scheme that enables verifiable inference for large language models (LLMs) without requiring the verifier to rerun the model or possess a powerful GPU. By binding each inference to a commitment stored in multivariate Terkle Trees, TCs detect tampering with only a 0.97% overhead for the prover and 0.12% for the verifier on LLaMA2. The approach improves robustness against tailored LLM attacks by up to 48% compared to previous methods that needed a verifier GPU.
arXiv:2606. 16352v1 Announce Type: cross Abstract: Computation integrity of remote large language model (LLM) serving can be questionable.
TensorCommitments (TCs) is a lightweight, tensor-native proof‑of‑inference scheme designed to enable verifiable inference for large language models (LLMs). It binds each LLM inference to a commitment using multivariate Terkle Trees, ensuring that any tampering breaks the commitment. For the LLaMA2 model, TC adds only 0.97% overhead for the prover and 0.12% for the verifier, while improving robustness to tailored attacks by up to 48% compared to prior work that requires a verifier GPU.
arXiv:2412. 12640v2 Announce Type: replace Abstract: The increasing demand for data privacy, alongside the benefits of aggregating data from networked devices, has catalyzed the emergence of federated learning (FL).
arXiv:2607. 13088v1 Announce Type: cross Abstract: Large Language Models (LLMs) are rapidly moving from research settings into the wild, deployed on enterprise infrastructure, personal devices, and edge platforms.
arXiv:2607. 13093v1 Announce Type: cross Abstract: On-device LLM inference faces a trilemma of response latency, limited hardware resources and user privacy.
arXiv:2606. 09551v1 Announce Type: cross Abstract: Two-server secure inference allows a client to query a hosted large language model (LLM) without revealing prompts or embeddings.
arXiv:2606. 00279v1 Announce Type: cross Abstract: Verifying claims about AI workloads is a pre- requisite for credible AI governance of covert adversaries (who comply with monitoring only when detection likelihood is high), yet the ap- parent non-determinism of GPU floating-point arithmetic forces auditors to accept approximate output matches.
arXiv:2609.40335v1 Announce Type: new Abstract: Differentially Private Stochastic Gradient Descent (DP-SGD) is a leading approach for privacy-preserving fine-tuning of large language models (LLMs). M...
Cocoon is a system architecture designed to enable efficient differentially private training by managing correlated noise across training iterations. The framework stores and processes large noise histories across CPU, GPU, and a memory extension module, optimizes sparse embedding tables, and utilizes near‑memory processing devices. Experiments on an FPGA‑based NMP prototype show performance improvements ranging from 1.23× to 10.82× compared to existing approaches.