The paper investigates how neural representations that use superposition can encode more features than their dimensionality. It derives sharp dimension thresholds for computing Boolean gates with a single threshold layer, distinguishing between two error criteria: a vanishing expected error count and high‑probability correctness of every output. The study reveals that the expected‑count threshold follows a ball geometry, while joint reliability follows a box geometry, and provides explicit thresholds for conjunction, disjunction, and majority gates, including a transition analysis for pairwise conjunction.
By Xiaoyu Li, Lequan Lin, Dai Shi, Jiaojiao Jiang, Junbin Gao, Andi Han
arXiv:2607. 21633v1 Announce Type: new Abstract: Logic Gate Networks (LGNs) implement computation through compositions of Boolean operations, yet unlike classical Boolean circuits, existing LGNs do not reliably benefit from increased depth.
By Taegun An, Dohun kim, Haebeom Lee, Changhee Joo
arXiv:2510. 04500v3 Announce Type: replace Abstract: This work demonstrates how increasing the number of neurons in a network without increasing its total number of non-zero parameters improves performance.
By Linghao Kong, Inimai Subramanian, Yonadav Shavit, Micah Adler, Dan Alistarh, Nir Shavit
arXiv:2609.09556v1 Announce Type: new
Abstract: Neural networks can leverage feature superposition to encode more concepts than dimensions, but cross-feature interference constrains the linear access...
By Enrico Vompa
The paper introduces mixed integer linear programming (MILP) formulations for training sparse disjunctive normal form (DNF) networks, which are small networks with a conjunctive hidden layer and a disjunctive output layer. By fixing the network structure and optimizing weights, the authors achieve globally optimized rule sets that balance misclassification costs against sparsity. Their enhanced formulation reduces integer decision variables and redundant constraints, and a combined initialization and dropout approach breaks weight symmetries to promote diverse, stable rule representations, leading to improved computational efficiency and competitive accuracy on synthetic and UCI datasets.
By Thomas Pammer, Johannes F\"urnkranz, Markus Sinnl
arXiv:2603. 07606v2 Announce Type: replace Abstract: Interpretable machine learning is essential in high-stakes domains where decision-making requires accountability, transparency, and trust.
By Hans Farrell Soegeng, Sarthak Ketanbhai Modi, Thomas Peyrin