The paper introduces a reference‑free diagnostic for physics‑informed neural networks that learn partial differential equation coefficients from noisy data. It presents three tests—an information‑matrix residual test, a heterogeneity statistic across window refits, and a Fisher‑rank statistic on the design—that can detect whether the operator used in the model is incorrect or whether the design is blind to certain rates, all without retraining the network. The authors demonstrate the method on a parabolic benchmark with 2 % noise, showing that the diagnostics reliably flag misspecified operators and blind designs while remaining silent when the model is correctly specified.
By Eric Fock
The paper introduces a reference‑free instrument that, from a single fit and without an oracle, can detect whether a hybrid PDE‑parameter estimator’s assumed operator is misspecified and distinguish this from mere parameter unidentifiability. In a self‑adjoint parabolic inverse problem, the proposed information‑matrix statistic correctly identifies misspecification with low false‑positive rates, while remaining silent when the design is correctly specified but non‑identifiable. The study demonstrates that conventional accuracy checks can miss significant operator errors, and it maps out the instrument’s blind spots and conditions under which its guarantees hold.
By Eric Fock
arXiv:2610.28155v1 Announce Type: cross
Abstract: Learned priors are used to reconstruct regions that measurements barely reach, and to report uncertainty there. They can fail silently: the reported...
By Mario Gianni
The paper investigates how adaptive experimentation can obscure errors in a model by repeatedly choosing settings where the incorrect model still fits well. It derives bounds on the power of adequacy tests based on the experimenter’s sampling distribution, introduces a computable diagnostic for unresolved error, and proposes a training‑free rule that allocates part of the design budget to falsification. Empirical tests show mixed results: the primary study fails detection, a secondary revision detects a modest fraction of misspecified campaigns, and performance varies with the time horizon and design choices.
By Ramnarayan Choudhary
arXiv:2606. 11267v1 Announce Type: new Abstract: Data leakage -- contamination of a model with information unavailable at baseline -- is the dominant reproducibility failure in machine-learning-based science, yet detection tools require training code, external data, or domain expertise.
By Laurence A. Jacobs
The paper audits whether routing entropy in Attention‑Residual transformer variants (Swin‑Tiny and DeiT‑Small) trained on CIFAR‑10/100 can signal prediction uncertainty beyond model confidence. Three tests examine the presence, consistency, and predictive power of routing signals, while a sensitivity audit measures how much injected effect the probes recover. Results show no significant improvement over confidence alone, with only modest recovery of injected signals and no consistent gains across seeds or metrics.
By Wenhao Liang, Lin Yue, Wei Emma Zhang, Mingyu Guo, Olaf Maennel, Weitong Chen
arXiv:2606. 25151v1 Announce Type: new Abstract: Physics-informed neural networks (PINNs) embed governing equations in their loss function, enabling mesh-free solutions to partial differential equations.
By David McShannon, Nicholas Dietrich
arXiv:2608. 13633v1 Announce Type: cross Abstract: Machine learning is now a central tool for solving inverse problems in particle physics and astronomy.
By Juan Cruz-Martinez, Carolina Cuesta-Lazaro, Alexander Held, Michael Kagan
arXiv:2607. 08456v1 Announce Type: cross Abstract: A model should refuse two different things: answers it would get wrong, and questions it should not answer at all, such as unanswerable ones or ones resting on a false premise.
By Benedikt J. Wagner
The study investigates how language models equipped with tools can still produce unsupported final claims, even when a single tool call could resolve the uncertainty. It defines two metrics—occurrence (how often unsupported claims arise) and conditional repair (how often they are fixed when evidence is provided). Experiments on Qwen3-32B and Gemma 4 show that providing the missing evidence consistently repairs all unsupported claims in the Qwen3-32B setup, while the Gemma 4 model never produced unsupported claims under the tested conditions.
By Justin Bronder
A 0.6B language model consistently answers YES to 1,200 logical tests, yet its behavior shows no discrimination. Linear probes reveal the correct verdict with high AUC (0.96) and transfer to unseen structures, but a single scalar readout fails due to a saturated decision threshold offset by +4.6 σ. Adjusting this threshold restores behavior accuracy from 50 % to 81 % and improves higher‑scale models, demonstrating that miscalibrated readouts, not hidden knowledge loss, drive performance gaps.
By Gnaneswar Villuri, Hashmath Shaik, Alex Doboli
arXiv:2608. 02786v1 Announce Type: new Abstract: AI systems can fail silently.
By Priyanka Bajaj (Independent Researcher)