arXiv Machine Learning
Calibrate-Then-Delegate: Safety Monitoring with Risk and Budget Guarantees via Model Cascades
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jun 8
Uncertainty-Guided Label Rebalancing for CPS Safety Monitoring
arXiv:2603. 25670v3 Announce Type: replace Abstract: Safety monitoring is essential for Cyber-Physical Systems (CPSs).
arXiv AI
Jun 30
Budgeted Act-or-Defer Multi-Agent LLM Deliberation with Local Reliability Bounds
arXiv:2606. 29654v1 Announce Type: new Abstract: Multi-agent deliberation among LLMs can improve reasoning, but deployment requires deciding when the current answer is reliable enough to act on and when it should be escalated to human review.
Hugging Face Trending Papers
Sep 2
ObserverBench: Testing Mechanistic Estimates for Intervention and Control
ObserverBench is a benchmark framework that evaluates whether internal mechanistic estimators—called observers—are suitable for guiding interventions, control, or safety actions in language models. It separates estimation accuracy from the loss incurred by the chosen action, demonstrating that accurate average estimates can still lead to poor decisions. Experiments on GPT‑2‑small, Qwen2.5‑7B, Gemma‑2‑9B‑it, and Qwen3.5‑9B show that observers trained on action loss tend to select lower‑loss actions, while traditional metrics like AUROC can rank monitors differently from deployment loss, highlighting the need for task‑specific evaluation.
arXiv AI
Sep 4
ObserverBench: Testing Mechanistic Estimates for Intervention and Control
ObserverBench is a benchmark framework that evaluates whether internal mechanistic estimators—called observers—are suitable for guiding interventions, control, or safety actions in language models. It separates estimation accuracy from the loss incurred by the chosen action, showing that accurate predictions do not always lead to better decisions. Experiments on GPT‑2‑small, Qwen2.5‑7B, Gemma‑2‑9B‑it, and Qwen3.5‑9B demonstrate that observers trained on action loss can reduce deployment loss, while traditional metrics like AUROC may rank monitors differently from actual performance.
arXiv Machine Learning
Oct 23
Scaling Laws for Safety Probes
The paper introduces two scaling laws that predict the performance of safety probes—tools used to monitor language models for harmful behavior—based on model size and the number of labeled examples. By creating six large safety datasets and training probes across multiple model families, the authors show that probe error decreases as a power law with model size and further improves with more examples, with the improvement rate increasing for larger models. These laws enable researchers to forecast probe accuracy and decide whether to invest in more data or a larger model to achieve a desired error threshold.
arXiv AI
Jun 15
Capability Minimization as a Safety Primitive: Risk-Aware Causal Gating for Least-Privilege LLM Agents
arXiv:2606. 13884v1 Announce Type: new Abstract: Modern decision systems increasingly rely on learned components whose outputs may be confident yet wrong, exposing downstream actions to costly errors.