The paper introduces a moving-horizon approximate branch‑and‑reduce method for training deep classification trees on large datasets with continuous features. It combines a hierarchical root‑subtree optimization framework, branch‑and‑reduce at the root, greedy heuristics for subtrees, and a low‑cost moving‑horizon refinement to improve accuracy. Experiments show the approach surpasses heuristic baselines in test accuracy while scaling better in dataset size and tree depth than existing global optimal solvers.
By Chenxuanyin Zou, Jiayang Ren, Qiangqiang Mao, Jing Liu, Marcus Lai, Yankai Cao
arXiv:2607. 28170v1 Announce Type: new Abstract: Optimal decision trees (ODTs) are compact, interpretable machine learning models that globally optimize a given objective, but their scalability remains challenging.
By Jacobus G. M. van der Linden, Mim van den Bos, Emir Demirovi\'c
arXiv:2607. 01417v1 Announce Type: new Abstract: Conditional inference trees (CIT) and conditional inference forests (CIF) reduce split-selection bias by testing features before choosing split thresholds, but repeated permutation tests and threshold searches can make these methods computationally expensive.
By Robert Milletich, Justin Downes, Steve Goley, Newel Hirst
arXiv:2608. 04310v1 Announce Type: new Abstract: The Rashomon effect describes the phenomenon that many models can achieve nearly equivalent performance on the same learning task, with significant ramifications for robustness, feature importance, and customizability.
By Zakk Heile, Hayden McTavish, Margo Seltzer, Cynthia Rudin
arXiv:2609.24741v1 Announce Type: new
Abstract: We develop an exact linear programming (LP) formulation for bounded-depth classification trees with binary features, using a junction-tree representati...
By Jiancheng TU, WenqiFan
arXiv:2604. 19753v2 Announce Type: replace Abstract: We propose a feature-free approach to algorithm selection: instead of hand-crafted instance features, we use pretrained text embeddings.
By Stefan Szeider
arXiv:2503. 12902v4 Announce Type: replace Abstract: Model trees provide an appealing way to perform interpretable machine learning for both classification and regression problems.
By Sabino Francesco Roselli, Eibe Frank
arXiv:2603.00326v2 Announce Type: replace
Abstract: Sparse oblique (SPO), part of the top-ranked configuration of Google's Yggdrasil Decision Forests (YDF), improve the accuracy while maintaining int...
By Ariel Lubonja, Jungsang Yoon, Haoyin Xu, Yue Wan, Yilin Xu, Richard Stotz, Mathieu Guillame-Bert, Joshua T. Vogelstein, Randal Burns
arXiv:2606. 11761v1 Announce Type: new Abstract: Dynamic data pruning techniques aim to reduce computational cost while minimizing information loss by periodically selecting representative subsets of input data during model training.
By Atif Hassan, Swanand Khare, Jiaul H. Paik
The paper introduces SIFT, a sample‑efficient framework for self‑improvement of coding agents that uses a fast tree‑search guided by an LLM‑as‑a‑judge signal. By performing pairwise comparisons of candidate patches and aggregating results with a regularized Bradley‑Terry model, SIFT limits expensive downstream evaluations to only the most promising nodes. The approach achieves higher coding performance on the Polyglot benchmark while reducing CPU hours, wall‑clock time, and API costs compared to prior tree‑search self‑evolution methods.
By Xinghong Fu, Aravinth Kulanthaivelu, Yutaro Yamada
arXiv:2606. 26337v1 Announce Type: new Abstract: Gradient Boosted Decision Trees (GBDT), exemplified by LightGBM, spend a dominant fraction of training time -- typically 65-70% -- constructing per-feature histograms.
By Yan Song
arXiv:2608. 03111v1 Announce Type: new Abstract: Double descent is commonly studied by scaling an explicit capacity parameter, such as neural-network width.
By Ryuichi Kanoh