BRIDGE is a model‑agnostic acceptance gate that filters model updates by cross‑fitting and bootstrap bounds, aiming to prevent false improvements that arise when validation gains do not translate to new data. Across multiple settings—including 11 new datasets, synthetic tasks, 12 retrospective clinical tasks, and 39 supporting datasets—BRIDGE reduces false‑improvement rates from roughly 19–25% with a naive rule to as low as 0–1.1%. The method also yields higher mean gains per deployed update, achieving 1.8 to 3.1 times the naive rule’s performance.
Models are updated repeatedly, and an update that looks better on validation data but not on new data is a false improvement that can misguide clinical decisions. The naive rule accepts every positive...
The paper introduces a benchmark of 400 synthetic healthcare workflow episodes to evaluate large language model (LLM) agents’ reliability in escalating care when evidence is missing or conflicting. It finds that while clean decision accuracy is high (0.97), models miss a large proportion of necessary escalations (32–97%) under uncertainty, and even the largest 9.7B model’s performance drops to 0.78 in conflict scenarios. A workflow-aware controller improves utility and reduces missed escalations, but its effectiveness is limited by lexical generalization failures.
arXiv:2608. 14551v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for title-abstract screening in systematic reviews, but their decisions lack calibrated uncertainty.
The study evaluates whether reward‑trained language models can learn to keep evidence‑sufficiency boundaries without a formal correctness signal. Using a controlled knock‑in design, a boundary‑keeping recipe (SFT followed by GRPO) dramatically increased spontaneous correct refusals from 25 % to 100 %, while a scrambled‑pairing placebo collapsed performance. The results show that final boundary‑keeping is trainable without an inference‑time verifier, though active numeric checking remains negligible.
arXiv:2607. 18828v1 Announce Type: new Abstract: Readiness stress-testing of medical AI has focused on closed-ended and multimodal benchmarks.
The paper investigates how to evaluate bounded model updates for ICU prediction models using the MIMIC‑IV database. After auditing the data for recording artifacts and cleaning it, the authors compare a bounded update that retrains only treatment‑context and fusion parameters with full fine‑tuning across two model families and two targets, finding that bounded updates are non‑inferior or superior in most cases but depend on target, model family, and learning‑rate tuning. The study also shows that audits should precede update evaluation and that the update boundary should be chosen per target.
arXiv:2608. 07813v1 Announce Type: new Abstract: An LLM judge deployed inside a reasoning pipeline does not merely measure quality, it decides which answer ships.
The paper presents a system for the MedReason 2026 challenge that tackles both multiple‑choice and open‑ended medical visual question answering using offline, containerized inference. Key findings include that comparing answer semantics rather than labels boosts retrieval‑only accuracy from 20.0 % to 57.5 % on a 200‑case holdout, and that varying the number of in‑prompt retrieved examples has minimal impact on final accuracy (93.5 %–94.0 %). The final system achieves 94.0 % MCQ accuracy on the development set and 93.20 % on the official pre‑evaluation, far surpassing the off‑the‑shelf baseline. "whyItMatters":"The results demonstrate that semantic‑aware retrieval and careful adapter tuning can dramatically improve medical VQA performance, offering a practical approach for high‑accuracy, offline inference in clinical settings."
The paper introduces a joint fact‑verification score that evaluates both answers and the evidence submitted with them. On the FEVEROUS dataset, replacing the DCUF evidence with UnifEE evidence improves the strict score by about 9.6 percentage points, while answer accuracy rises only 1.96 points. The study also shows that increasing context length for large language models yields modest evidence‑gain improvements, and that detailed answer‑evidence analyses uncover patterns missed by aggregate metrics.
arXiv:2603. 10494v2 Announce Type: replace-cross Abstract: Evidence-grounded generation produces summaries whose claims should be supported by supplied evidence, but claim-level verifiers provide noisy feedback and can reward models that simply say less.
arXiv:2608. 14639v1 Announce Type: cross Abstract: Per-field accept/review with selective risk at most alpha -- accept a field only if the error rate among accepted fields is controlled -- is the trust contract document-extraction systems need, and the natural procedure silently violates it on real documents.
The paper introduces TrustSwap, a counterfactual test that swaps or removes source reliability labels while keeping evidence text constant, to evaluate how retrieval‑augmented fact‑checking models respond across verdict, confidence, and search decisions. Experiments on untrained and RL‑trained models show that confidence and search largely follow labels, yet label changes can flip a significant portion of verdicts, especially in larger models. The authors propose trust‑swap augmentation (TSA) to mitigate this shortcut, demonstrating reduced verdict flip rates and maintained accuracy in several settings, though its effectiveness diminishes at larger model scales.