arXiv Machine Learning By Parivesh Priye, Yufeng Wang, Haibin Ling, Michael Chaykowsky

Available Guardrails: Certifying Selective Prediction across ML Systems

Read the original on arXiv Machine Learning →

The paper introduces a method to certify selective prediction in machine learning systems by computing the availability of safety gates through exact-binomial inversion and dynamic programming. It demonstrates that a truth-informed planner can significantly improve mean coverage over naive approaches, and that reallocating error budgets further enhances coverage across diverse applications such as LLM tool‑calling, content moderation, lesion classification, and recommendation. The study highlights the importance of planning and finite‑sample estimation in ensuring reliable, granular deployment of selective predictors.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
Sep 14

Quality-Constrained Routing over a Fixed Pool of Quantized Mixture-of-Experts Instances

The paper proposes a method for routing requests to a fixed pool of quantized Mixture-of-Experts (MoE) instances, aiming to maximize throughput while respecting a quality‑degradation budget. It introduces Fragility‑Weighted Perplexity (FWP) as a request‑specific risk metric derived from prompt tokens, and uses a window‑level linear program to compute a reduced‑reward score that aligns with the LP optimum. Experiments on Qwen prompts show that FWP‑based allocation improves throughput by 2.5% over request‑agnostic mixing and static configurations.

By Zhenghong Huang, Hongfan Wu, Jiheng Zhang
arXiv Computation and Language
Aug 21

Stopping and Routing LLM Judge Panels

arXiv:2608. 19802v1 Announce Type: new Abstract: LLM evaluation pipelines often have many candidate judges: general LLM-as-a-judge prompts, reward models, safety classifiers, confidence variants, and task-specific verifiers.

By Bin Zhu, Yi Xie, Yanghui Rao
arXiv AI
Jul 1

Qualified Educational Capacity Planning under Heterogeneous Student Support Needs: A Synthetic Benchmark and Decision-Support Framework

arXiv:2606. 30650v1 Announce Type: cross Abstract: Educational support services often face a qualified-capacity problem: staff time is scarce, qualifications decay, new support needs can appear before anyone is prepared for them, and training consumes the same hours needed by current students.

By Carlos Eduardo Sanoja, Oscar Enrique Moreno Mayz
arXiv AI
Sep 15

When Should a World Model Move? Loss-Conditioned State Execution

The paper introduces loss‑conditioned state execution, a model‑agnostic technique that decides whether to apply a world model’s proposed state change or keep the current state based on whether the change reduces downstream loss. It formalizes state movability as the existence of a loss‑reducing feasible correction and constructs loss‑specific proposals from predictive distributions, executing them only when a groupwise lower confidence bound on loss improvement is positive. Experiments on forecasting and dynamics benchmarks show that the method accepts updates for a subset of cases, achieving lower bounded loss than persistence or always executing the proposal, and highlights that event predictability and loss‑based decisions must be evaluated separately.

By Jintao Xu, Zhengyu Chen, Ben Zhang, Yongzhi Qi, Jianshen Zhang