arXiv Machine Learning
6d ago

Optimizing Sparse Boolean Rule Networks using Mixed Integer Linear Programming

The paper introduces mixed integer linear programming (MILP) formulations for training sparse disjunctive normal form (DNF) networks, which are small networks with a conjunctive hidden layer and a disjunctive output layer. By fixing the network structure and optimizing weights, the authors achieve globally optimized rule sets that balance misclassification costs against sparsity. Their enhanced formulation reduces integer decision variables and redundant constraints, and a combined initialization and dropout approach breaks weight symmetries to promote diverse, stable rule representations, leading to improved computational efficiency and competitive accuracy on synthetic and UCI datasets.

By Thomas Pammer, Johannes F\"urnkranz, Markus Sinnl
arXiv AI
Sep 15

Certifiably Interpretable Training of ReLU-MLPs for Boolean Tasks with Guaranteed Truth-Table Generalization

The paper introduces MACCHIATO, a training algorithm that builds a ReLU‑MLP from partial truth‑table data while simultaneously constructing an explicit Boolean circuit over AND, OR, and XOR gates that certifies the network’s computation. The method iteratively projects residuals onto low‑dimensional Boolean classes, compiles the resulting circuit into a ReLU‑MLP, and uses logic minimization and influence‑based variable selection to achieve a six‑layer network with provable truth‑table error bounds. Experiments on synthetic random‑junta tasks show that these certified networks outperform Adam‑trained MLPs in data‑sparse or projection‑aligned regimes and complete faster than flat ESPRESSO in certain settings.

By Hrad Ghoukasian, Anastasis Kratsios