Hugging Face Trending Papers
Lifetime-valid decisions by continually evolving AI analysts
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
3d ago
Lifetime-valid decisions by continually evolving AI analysts
The paper introduces a method called continual validation for AI analysts that evolve over time. By evaluating each new version on simulated null data before deployment, it guarantees that the lifetime probability of exceeding a target Type I error rate stays bounded, regardless of how many updates are made. Experiments on bioequivalence studies and A/B tests show that continual validation reduces Type I error rates from 0.098 and 0.183 down to at most 0.045 and 0.044 while preserving most of the statistical power.
arXiv Statistics ML
4d ago
Adaptive auditing of AI systems with anytime-valid guarantees
The paper introduces a hypothesis testing framework for adaptive auditing of AI systems, addressing the challenge of limited, opportunistically collected data. It defines two competing null hypotheses—one from the model’s perspective and one from the auditor’s—and uses Safe Anytime-Valid Inference (SAVI) to perform betting-based tests that yield simultaneous e-processes. The authors prove that, with a powerful auditor, the two hypotheses become asymptotically inverse, allowing a global robustness certification, and demonstrate empirically that their methods control Type‑I error and can reach conclusions with as few as 20 observations.
arXiv AI
Aug 25
The Greatness of Science Cannot Be Planned: Agentic Auto-Research is Fuzz Testing
The article argues that agentic auto‑research should be guided by dense, intermediate signals of epistemic progress rather than by sparse final benchmarks. It compares this approach to fuzz testing, where coverage provides continuous feedback that directs input mutation. The authors propose controlled experiments to test whether such signals improve discovery efficiency and reduce false positives, and demonstrate in a simulated physics setting that an AI agent using feedback‑driven search uncovers a hidden law while optimization‑driven baselines fail.
arXiv AI
Sep 30
Which Self-Improvements Should We Trust? Reliable Self-Improvement When Agents Reuse Their Benchmarks
arXiv:2609.33180v2 Announce Type: replace-cross Abstract: As recursive self-improvement (RSI) rapidly advances, reliable evaluation becomes critical for guiding adaptive search. RSI typically relies...
arXiv AI
Jun 9
PACE: Anytime-Valid Acceptance Tests for Self-Evolving Agents
arXiv:2606. 08106v1 Announce Type: new Abstract: Self-evolving agents improve by repeatedly proposing changes to their own prompts, skills, or workflows and keeping those that score higher on a small held-out set.
arXiv AI
Aug 11
Agentic Auto-Research is Fuzz Testing
arXiv:2608. 09855v1 Announce Type: new Abstract: Autonomous research agents can generate experiments faster than researchers can validate them.