arXiv Machine Learning By Jiahe Li, Omar Melikechi

Just add noise: Debiasing tree-based variable importance in mixed data

Read the original on arXiv Machine Learning →

The paper investigates the bias in variable importance scores produced by tree-based methods, noting that continuous predictors are favored over categorical ones. It offers a theoretical explanation for this bias and proposes a straightforward fix: adding a small amount of noise to each categorical predictor. The authors validate the correction on both simulated and real-world datasets and integrate it with integrated path stability selection to achieve variable selection with false discovery control for mixed data.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
Jun 10

Correcting Variable Importance Scored by Random Forests

arXiv:2606. 10770v1 Announce Type: cross Abstract: Variable importance produced by Random Forests (RF) is used widely in statistical data analysis, and has played an important role in a variety of tasks such as assisting model interpretation, model selection and diagnosis, and cost-bounded learning etc.

By Guancheng Zhou, Haiping Xu, Jason Liu, Donghui Yan
arXiv Machine Learning
Aug 4

Beyond Noise: A Hypothesis Testing Approach to Robust Feature Selection

arXiv:2511. 20851v3 Announce Type: replace-cross Abstract: Feature selection remains difficult in modern high-dimensional settings, and established methods such as Boruta and Recursive Feature Elimination are either computationally costly or lack a statistically justified stopping criterion for their importance scores.

By Mousam Sinha, Tirtha Sarathi Ghosh, Koushik Biswas, Ridam Pal
arXiv Machine Learning
Jul 3

Conditional Inference Trees and Forests for Feature Selection

arXiv:2607. 01417v1 Announce Type: new Abstract: Conditional inference trees (CIT) and conditional inference forests (CIF) reduce split-selection bias by testing features before choosing split thresholds, but repeated permutation tests and threshold searches can make these methods computationally expensive.

By Robert Milletich, Justin Downes, Steve Goley, Newel Hirst