The paper introduces CELLAUDIT, a method for auditing whether inputs claimed to influence predictive models actually do so. By testing if an input can enter the computation, whether predictions depend on it, and if that dependence improves observed responses, the authors evaluate agent-generated predictors on a morphology‑transcriptomics benchmark (BBBC047). Their findings show that many models claim compound contributions that are not supported by the data, and that falsification‑guided revisions can recover genuine input effects while improving performance.
By Mengran Li, Bo Li, Chengyang Zhang, Yang Yan, Jinfeng Xu, Zhenchao Tang
FinForesight is a diagnostic backtesting benchmark designed to evaluate deep research systems for financial forecasting. It tests four forecasting tasks—valuation, return ranking, event direction, and future financial fundamentals—across Chinese, U.S., and Hong Kong markets, while also including two diagnostic tasks for counterfactual reasoning and financial calculation. The framework limits evidence to what was available at the forecast date and compares pretraining-era, post-cutoff, and live evaluations to assess how well controlled backtests predict future performance.
By Fengyuan Liu, Mukai Li, Jieshun You, Yilin Zheng, Chenjia Min, Yizhou Zhang, Harper Chen, Qi Liu
arXiv:2608. 08126v1 Announce Type: new Abstract: Credit scoring increasingly relies on models whose decision logic cannot be read off their parameters, in tension with supervisory expectations that adverse decisions be explainable.
By Gregorius Reynaldi Pratama, Kuo-Kun Tseng
The paper introduces a framework that distinguishes two causes of saturation in clinical prediction: a learner gap, where the model fails to use available information, and a measurement‑channel ceiling, where the recorded variables limit performance. It provides theoretical characterizations, finite‑sample diagnostics, and empirical audits across three large cohorts, showing that well‑tuned models approach the frontier while deficient learners leave large gaps. A PRISMA‑guided synthesis across 104 tasks reveals consistent channel‑level patterns, suggesting that improving the learner or the measurement channel can audit and potentially lift performance.
By Sayeed Shafayet Chowdhury, Nusrat Jahan, Snehasis Mukhopadhyay, Shiaofen Fang, Vijay R. Ramakrishnan
arXiv:2608. 16147v1 Announce Type: new Abstract: Class-imbalance handling is routinely evaluated on a single benchmark dataset, and the resulting conclusions are reported as if they were properties of the method.
By Diyorbek Musaev
arXiv:2609.37223v1 Announce Type: new
Abstract: Credit-risk prediction is important in banking, but a prediction alone does not explain why an applicant is risky or how it should be combined with oth...
By Aakash Kumar Tiwari
arXiv:2607. 19453v1 Announce Type: cross Abstract: We audit whether candle-based machine-learning models can turn predictions of cryptocurrency extrema or short-horizon outcomes into positive Binance Spot paper policies after assumed costs.
By Ayoub Jadouli
arXiv:2608. 19760v1 Announce Type: cross Abstract: Audited against causal ground truth from executed replay in a single-agent tool environment (ALFWorld), none of the step-level credit signals used to train LLM agents -- LLM-judge scores, outcome-conditioned logprob ratios, or the policy's own confidence -- identifies which steps causally matter better than chance.
By Haiyue Zhang
arXiv:2609.24677v1 Announce Type: new
Abstract: Large language models (LLMs) are increasingly used to make predictions from numerical time-series histories and textual events. Yet accuracy alone cann...
By Jie Gong, Maowei Jiang, Zhiwei Liu, Yankai Chen, Guojun Xiong, Xue Liu, Min Peng, Qianqian Xie, Sophia Ananiadou
arXiv:2608. 14903v1 Announce Type: new Abstract: Quantitative forecasts of frontier artificial intelligence often connect dated targets to trends in benchmark scores, training compute, release time, or expert belief.
By Fabricio F Costa
The paper introduces a formal framework and benchmark for time‑series world models (TSWMs) that separates state, actions, and exogenous inputs, and defines a new metric called mechanism consistency to evaluate whether model predictions move in the expected direction when actions change. Experiments on eight public datasets show that using a frozen latent prediction space and gated output fusion improves prediction accuracy, while prediction error and mechanism consistency often diverge, with the best‑performing models sometimes failing to exhibit consistent directional responses. Adding a directional supervision loss significantly boosts mechanism consistency without affecting mean‑absolute error, providing a practical recipe for building more reliable TSWMs.
By Haochen Zhang, Jiaheng Guo, Zhen Xu, Zachary Plotkin, Nicholas Konz, Zhen Tan, Tianlong Chen
arXiv:2608.30086v1 Announce Type: cross
Abstract: Credit-default prediction is an important task in financial decision making. Traditional methods use fitted classifiers such as logistic regression a...
By Rishi Datta, Lavanya Prahallad