arXiv Machine Learning
5d ago

Neyman Orthogonal Activation Steering

The paper introduces a new framework for activation steering in large language models, distinguishing between the steering direction (estimand) and the method used to recover it (estimator). It shows that most existing techniques can be viewed as plug‑in estimators of the average gradient of an outcome regression, and proposes a complementary estimator based on the Stein score to directly target the gradient. By combining these approaches, the authors achieve an orthogonalized estimator with double‑robustness properties and demonstrate its effectiveness on benchmark datasets.

By David Arbour, Erfan Jahanparast, Serina Chang, Avi Feller
arXiv Machine Learning
Sep 1

Prediction-Powered Conditional Inference

arXiv:2603.05575v2 Announce Type: replace-cross Abstract: We study prediction-powered conditional inference in the setting where labeled data are scarce, unlabeled covariates are abundant, and a blac...

By Yang Sui, Jin Zhou, Hua Zhou, Xiaowu Dai
arXiv Machine Learning
Jun 15

Minimum Distance Summaries for Robust Neural Posterior Estimation

arXiv:2602. 09161v2 Announce Type: replace-cross Abstract: Simulation-based inference (SBI) enables amortized Bayesian inference by first training a neural posterior estimator (NPE) on prior-simulator pairs, typically through low-dimensional summary statistics, which can then be cheaply reused for fast inference by querying it on new test observations.

By Sherman Khoo, Dennis Prangle, Song Liu, Mark Beaumont