Decision trees generate interpretable if--then rules, yet they contain irrelevant conditions (IRCs). These IRCs arise from the structural mechanism of tree splitting and persist even in modern optimal sparse tree induction algorithms.
arXiv:2607. 13874v1 Announce Type: new Abstract: Decision trees generate interpretable if--then rules, yet they contain irrelevant conditions (IRCs).
The paper investigates four ways to grow a classifier—adding a tree level, a hidden unit, a leaf split, and a statistically significant split—under a fixed protocol for tree‑structured and constructive models. It shows that the most natural method of deepening a soft decision tree by duplicating a leaf’s class distribution leaves the gradient of new gates identically zero, preventing learning, and proposes a small random perturbation as a fix. The other three growth decisions each provide a distinct benefit: fitting a new hidden unit to residual error yields a smaller network, splitting the leaf with the largest expected error adds sparsity, and requiring statistical significance before splitting adds no value and reduces accuracy.
arXiv:2605. 20716v5 Announce Type: replace Abstract: Random forests construct each tree with a different, randomised representation of the feature space.
RCProb is a probabilistic extension of rule extraction from tree ensembles that improves probability estimates by using smoothed atomic class-conditional evidence and a support‑adaptive mixture for final rule probabilities. Compared to RuleCOSI+, RCProb reduces median paired log‑loss by 71.9% for random forests and 62.5% for gradient boosting, while also decreasing the number of extracted rules by about 38% for both ensemble types. The method shows significant improvements in calibration metrics such as Confidence‑ECE and competitive native probability estimates, with further gains possible through post‑hoc calibration.
arXiv:2606. 30995v1 Announce Type: new Abstract: Recent work has shown that well-optimized individual decision trees can match complex black box models in some settings, primarily in noisy domains.
arXiv:2605. 22740v2 Announce Type: replace Abstract: Decision trees assign identical confidence to instances near and far from each split threshold.
The paper presents a model‑agnostic framework that performs constrained post‑hoc error correction for binary classifiers. It searches for an interpretable conjunction of feature–threshold rules that corrects remaining false positives or false negatives while limiting newly introduced errors, using graph‑based search, depth‑dependent constraints, and a reduced‑histogram threshold evaluation. Experiments on a large binary‑classification problem show that the method can efficiently identify compact correction rules, such as a configuration that removes 90% of false positives while only sacrificing 5% of true positives.
arXiv:2608. 08674v1 Announce Type: new Abstract: In the operation of machine learning models, model update is a fundamental process that requires careful consideration of its impact on downstream decision-making.
The paper proposes treating leaf values of a gradient‑boosted ensemble as coordinates in ℝ^M, turning the model into a linear function over these coordinates. This perspective allows exact contrastive explanations: the difference between two instances is a vector that is zero wherever they share a leaf, so the gap is attributed to a few coordinates linked to specific tree splits. The authors build a recourse method based on this representation, achieving near‑perfect reconstruction of the model’s decision and demonstrating competitive performance on tabular datasets, especially when recommendations are limited to actionable changes.
arXiv:2608. 15725v1 Announce Type: new Abstract: Predictive models in clinical and regulated settings must be accurate and fully auditable.
arXiv:2409. 12788v3 Announce Type: replace Abstract: Recently there has been a surge of interest in optimal decision tree (ODT) methods that globally optimize accuracy directly, in contrast to traditional approaches that locally optimize an impurity or information metric.
The paper introduces DICS, a clustering-based framework that uses data-informed priors to construct a compact set of candidate splits for decision tree classifiers. By incorporating class-aware structure, DICS reduces the split search space, preserving predictive performance while cutting training time. The authors provide theoretical analysis and experimental results showing comparable accuracy to exhaustive search across synthetic and benchmark datasets.