arXiv AI By Alexander Okezue Bell

Hidden-State Privacy Has an Empty Middle

Read the original on arXiv AI →

arXiv:2605. 24042v3 Announce Type: replace-cross Abstract: Of $1{,}536$ Gaussian release covariances we tested for single-layer hidden-state privacy, zero achieve both moderate utility and moderate privacy against an adaptive retrieval attacker.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning
Aug 28

Privacy Without Regret: Differentially Private Inference-Time Alignment

The paper introduces Private Best-of-N (PrivBoN), a method that adds calibrated Gumbel noise to reward scores during inference-time alignment, achieving both ε-differential privacy and KL-regularized alignment. When the privacy budget exceeds a critical threshold ε*, the noise becomes regret-optimal, matching the theoretical alignment skyline. The authors also propose Private Inference-Time Pessimism (PrivITP), which uses χ^2-regularized rejection sampling and a two-phase Gaussian mechanism to provide ex-post (ε,δ)-DP with a privacy cost independent of the number of responses, and demonstrate that both methods outperform standard Best-of-N across multiple models and datasets.

By Ishi Jain, Nandini Bhattad, Sayak Ray Chowdhury
arXiv AI
Sep 10

PAC-Private Autoregressive Generation: Calibrating Noise to Ensemble Disagreement

The paper introduces PAC‑Private Autoregressive Generation, a method that calibrates noise based on ensemble disagreement across overlapping ‘worlds’ of a private corpus, thereby extending PAC privacy from classification to text generation. By training adapters on a frozen public model and using posterior‑weighted disagreement to add noise only when predictions vary, the approach achieves strong privacy guarantees while preserving most of the fine‑tuning benefit. Experiments on WikiText‑103 with GPT‑2‑small show 74 % of the fine‑tuning gain retained with a per‑token budget of 2⁻³², and membership‑inference success bounded to 51.08 % after one million tokens, outperforming PMixED under matched conditions.

By Mina Mirzadehsarcheshmeh, Amir Keyvan Khandani