The Flow has not summarised this story yet — read it at arXiv Machine Learning.
The paper introduces a reference‑free diagnostic for physics‑informed neural networks that learn partial differential equation coefficients from noisy data. It presents three tests—an information‑matrix residual test, a heterogeneity statistic across window refits, and a Fisher‑rank statistic on the design—that can detect whether the operator used in the model is incorrect or whether the design is blind to certain rates, all without retraining the network. The authors demonstrate the method on a parabolic benchmark with 2 % noise, showing that the diagnostics reliably flag misspecified operators and blind designs while remaining silent when the model is correctly specified.
The paper introduces a reference‑free instrument that, from a single fit and without an oracle, can detect whether a hybrid PDE‑parameter estimator’s assumed operator is misspecified and distinguish this from mere parameter unidentifiability. In a self‑adjoint parabolic inverse problem, the proposed information‑matrix statistic correctly identifies misspecification with low false‑positive rates, while remaining silent when the design is correctly specified but non‑identifiable. The study demonstrates that conventional accuracy checks can miss significant operator errors, and it maps out the instrument’s blind spots and conditions under which its guarantees hold.
arXiv:2610.28155v1 Announce Type: cross Abstract: Learned priors are used to reconstruct regions that measurements barely reach, and to report uncertainty there. They can fail silently: the reported...
The paper investigates how adaptive experimentation can obscure errors in a model by repeatedly choosing settings where the incorrect model still fits well. It derives bounds on the power of adequacy tests based on the experimenter’s sampling distribution, introduces a computable diagnostic for unresolved error, and proposes a training‑free rule that allocates part of the design budget to falsification. Empirical tests show mixed results: the primary study fails detection, a secondary revision detects a modest fraction of misspecified campaigns, and performance varies with the time horizon and design choices.
arXiv:2606. 11267v1 Announce Type: new Abstract: Data leakage -- contamination of a model with information unavailable at baseline -- is the dominant reproducibility failure in machine-learning-based science, yet detection tools require training code, external data, or domain expertise.
The paper audits whether routing entropy in Attention‑Residual transformer variants (Swin‑Tiny and DeiT‑Small) trained on CIFAR‑10/100 can signal prediction uncertainty beyond model confidence. Three tests examine the presence, consistency, and predictive power of routing signals, while a sensitivity audit measures how much injected effect the probes recover. Results show no significant improvement over confidence alone, with only modest recovery of injected signals and no consistent gains across seeds or metrics.