arXiv:2609.17545v1 Announce Type: new Abstract: Deep learning models for cervical cytology are almost always evaluated as if every prediction must be acted upon, yet a screening system deployed along...
The paper proposes a method for selectively querying language‑model advice in reinforcement learning by predicting the value of potential responses and only querying when the expected benefit outweighs the cost. It introduces a certified, response‑contingent metareasoning framework that guarantees near‑optimal advice usage under certain assumptions, and demonstrates that a calibrated controller with Qwen2.5 advisors can improve task performance while drastically reducing the number of advice calls on the BabyAI benchmark.
arXiv:2509. 21514v4 Announce Type: replace Abstract: Research on Knowledge Tracing (KT) models traditionally focuses on improving predictive accuracy.
arXiv:2606. 04161v1 Announce Type: new Abstract: Different predictors often excel on different inputs, so picking the best one per instance promises higher accuracy than committing to a single model.
arXiv:2608. 07913v1 Announce Type: cross Abstract: Selective-risk certificates promise that accepted outputs meet a declared error target.
The paper introduces a distribution‑free certification layer that can be applied to any crash‑severity prediction model without modifying the model itself. It provides guarantees for ordinal outcomes, per‑class validity, transfer of coverage to unobserved severities, and one‑sided certificates under deployment shift, all grounded in a functional of the true data law. The framework is evaluated on 5.2 million Texas records, demonstrating a model‑independent lower bound on set width for vulnerable road users and is released as an open‑source package with theorem‑level tests.
arXiv:2608. 01378v1 Announce Type: new Abstract: Design campaigns in chemistry, materials science, and machine learning share a bottleneck: determining how good a candidate truly is requires an expensive evaluation - an experiment, a first-principles simulation, or a full training run.
The paper introduces a method to certify selective prediction in machine learning systems by computing the availability of safety gates through exact-binomial inversion and dynamic programming. It demonstrates that a truth-informed planner can significantly improve mean coverage over naive approaches, and that reallocating error budgets further enhances coverage across diverse applications such as LLM tool‑calling, content moderation, lesion classification, and recommendation. The study highlights the importance of planning and finite‑sample estimation in ensuring reliable, granular deployment of selective predictors.
arXiv:2608. 05064v1 Announce Type: cross Abstract: Small open-weight language models increasingly run in private, offline, and cost-sensitive settings, where the key deployment question is not only what a model answers but when it should defer to a human.
Enterprise practitioners read agent leaderboards as if they ranked agent capability. We show, across three open agent-trace benchmarks (TheAgentCompany, $τ^2$-bench, and AppWorld), that the agent main effect accounts for less than 3% of total variance in every dataset and check type, while the agent-by-task interaction accounts for 7-23%.
The study audited seven public educational prediction datasets using four pre‑modeling reliability checks—baseline gap, split instability, null separation, and metadata adequacy under group‑aware holdout. Only three datasets passed all checks; the others failed either group‑aware generalization tests or lacked necessary provenance metadata. The audit revealed that cross‑group fragility, rather than weak iid performance, was the dominant failure mode, and that increasing model complexity did not resolve these structural issues.
arXiv:2607. 19442v1 Announce Type: cross Abstract: Machine unlearning is commonly evaluated by matching a retrained oracle on trained probes.