The paper introduces TOP‑Gain, a new criterion for Random‑Forest wrapper feature selection that uses truncated out‑of‑bag probability gain to reduce noise in feature scoring. By truncating negative probability changes, TOP‑Gain cuts roughly two‑thirds of noise variance while keeping signal, leading to more reproducible feature rankings. In extensive benchmarks on 26 OpenML‑CC18 datasets, TOP‑Gain outperformed 13 of 14 baselines in ROC‑AUC, accuracy, and Macro‑F1, matched VSURF’s accuracy with far less computation, and produced more stable feature sets without sacrificing accuracy.
Random-Forest wrapper feature selection can use out-of-bag (OOB) predictions as a built-in validation signal, but the criteria that decide each forward step accumulate OOB estimation noise. We propose...
arXiv:2511. 20851v3 Announce Type: replace-cross Abstract: Feature selection remains difficult in modern high-dimensional settings, and established methods such as Boruta and Recursive Feature Elimination are either computationally costly or lack a statistically justified stopping criterion for their importance scores.
arXiv:2607. 01417v1 Announce Type: new Abstract: Conditional inference trees (CIT) and conditional inference forests (CIF) reduce split-selection bias by testing features before choosing split thresholds, but repeated permutation tests and threshold searches can make these methods computationally expensive.
arXiv:2607. 26964v1 Announce Type: cross Abstract: We study feature bagging through the lens of algorithmic stability.
We study feature bagging through the lens of algorithmic stability. Feature bagging is an ensemble strategy that aggregates base learners trained on randomly subsampled feature subsets, possibly in a data-dependent manner.
arXiv:2605. 20716v5 Announce Type: replace Abstract: Random forests construct each tree with a different, randomised representation of the feature space.
arXiv:2608. 14866v1 Announce Type: cross Abstract: Objective: Small-sample molecular classification requires feature selectors that identify predictive, stable, and nonredundant subsets for binary and multiclass outcomes.
arXiv:2609.24126v1 Announce Type: cross Abstract: Black-box machine learning models increasingly deliver strong predictions, but extracting useful information from them, such as a set of important fe...
arXiv:2610.33909v1 Announce Type: cross Abstract: Out-of-distribution (OOD) detection identifies inputs outside the training distribution. Rank-based analyses such as RankFeat and RankWeight suggest...
arXiv:2606. 01566v1 Announce Type: new Abstract: Small-to-medium scientific datasets place machine learning pipelines under two compounding pressures.
arXiv:2512. 08216v4 Announce Type: replace-cross Abstract: Accurate segmentation of lung tumors from 3D computed tomography (CT) scans is essential for automated treatment planning and response assessment.
arXiv:2606. 03549v1 Announce Type: new Abstract: Hyperparameter optimization (HPO) for Random Forest faces a specific difficulty in tuning the number of trees: the predictive score typically improves monotonically with ensemble size, so standard methods such as Tree-structured Parzen Estimator (TPE) and Hyperband require a predefined search range and often drive the estimate toward its right boundary.