OpenAI Blog
Semi-supervised knowledge transfer for deep learning from private training data
Read the original on OpenAI Blog →The Flow has not summarised this story yet — read it at OpenAI Blog.
Related stories
arXiv Machine Learning
4d ago
High-Dimensional Asymptotics and Dataset Selection for Private Transfer Learning
The paper addresses the challenge of selecting external datasets for private transfer learning by modeling high‑dimensional regression with heterogeneous sources and a weighted ridge estimator. It relies solely on aggregated statistics and offers privacy guarantees under $ ho$‑zero‑concentrated differential privacy for labels or both features and labels. A deterministic equivalent of test error is derived, enabling optimization of hyperparameters and decision‑making about the utility of private external data without accessing individual records.
arXiv Machine Learning
Jun 26
Escaping Iterative Parameter-Space Noise: Differentially Private Learning with a Hypernetwork
arXiv:2606. 26772v1 Announce Type: new Abstract: Differentially private (DP) training of neural networks is often hindered by the large amount of noise required by gradient-based methods such as DP-SGD, which repeatedly inject high-dimensional noise in parameter space throughout training.
arXiv Machine Learning
Jul 23
Differentially Private Neural Network Training Under the Hidden State Assumption
arXiv:2407. 08233v3 Announce Type: replace Abstract: Current differentially private learning paradigms face a severe utility bottleneck: DP-SGD degrades performance through noise accumulation over training steps, while aggregation-based approaches such as PATE suffer from data inefficiency due to disjoint data partitioning.
arXiv AI
Jul 9
Collaborative Synthetic Data Generation for Knowledge Transfer in Federated Learning
arXiv:2607. 07565v1 Announce Type: cross Abstract: One-shot federated learning (OSFL) addresses the communication overhead of federated learning by limiting training to a single round, but doing so without sacrificing model quality is non-trivial, particularly when client data distributions diverge.
arXiv Machine Learning
Jul 23
End-to-End Differential Privacy in Training Deep Neural Network Classifiers
arXiv:2607. 19580v1 Announce Type: new Abstract: Differentially private machine learning enables model training on sensitive data while ensuring that individual data is unlikely to be recoverable from the parameters of the resulting model.
arXiv Machine Learning
Jun 8
Model Recycling Framework for Multi-Source Data-Free Supervised Transfer Learning
arXiv:2508. 02039v2 Announce Type: replace Abstract: Increasing concerns for data privacy and other difficulties associated with retrieving source data for model training have created the need for source-free transfer learning, in which one only has access to pre-trained models instead of data from the original source domains.