← Back to all news
arXiv AI By Shashie Dilhara Batan Arachchige, Robin Carpentier, Hassan Jameel Asghar, Dali Kaafar

SpliTEE: Improving LLM Inference on Trusted Hardware with Differentially Private GPU Outsourcing

Read the original on arXiv AI →

SpliTEE extends the split‑inference architecture of Slalom to large language models by protecting intermediate GPU computations with differential privacy rather than encryption. The authors show that masking intermediate representations is essential, as a prompt‑reconstruction attack can recover prompts with about 80% accuracy. Their global sensitivity analysis bounds the noise needed, and they demonstrate that SpliTEE on Intel TDX achieves near‑double the speed of fully CPU‑based inference and outperforms encryption‑based Slalom while maintaining higher accuracy.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Oct 2

TensorCommitments: A Lightweight Verifiable Inference for Language Models

TensorCommitments (TCs) is a lightweight, tensor-native proof‑of‑inference scheme that enables verifiable inference for large language models (LLMs) without requiring the verifier to rerun the model or possess a powerful GPU. By binding each inference to a commitment stored in multivariate Terkle Trees, TCs detect tampering with only a 0.97% overhead for the prover and 0.12% for the verifier on LLaMA2. The approach improves robustness against tailored LLM attacks by up to 48% compared to previous methods that needed a verifier GPU.

By Oguzhan Baser, Elahe Sadeghi, Eric Wang, Nico Vergauwen, Sam Kazemian, Hong Kang, Sandeep P. Chinchali, Sriram Vishwanath
llmssafety
More like this →
arXiv AI
Oct 26

TensorCommitments: A Lightweight Verifiable Inference for Large Language Models

TensorCommitments (TCs) is a lightweight, tensor-native proof‑of‑inference scheme designed to enable verifiable inference for large language models (LLMs). It binds each LLM inference to a commitment using multivariate Terkle Trees, ensuring that any tampering breaks the commitment. For the LLaMA2 model, TC adds only 0.97% overhead for the prover and 0.12% for the verifier, while improving robustness to tailored attacks by up to 48% compared to prior work that requires a verifier GPU.

By Oguzhan Baser, Elahe Sadeghi, Eric Wang, Nico Vergauwen, Sam Kazemian, Hong Kang, Sandeep P. Chinchali, Sriram Vishwanath
llmssafety
More like this →