arXiv Machine Learning By Boldsaikhan Bolorkhuu, Tsogtgerel Gantumur

Exact ReLU realization of binary affine refinement iterates via reflection folding and cone switching

Read the original on arXiv Machine Learning →

The paper investigates vector‑valued binary affine refinement operators with finite matrix masks and compactly supported continuous piecewise‑linear data. It demonstrates that every finite refinement iterate can be exactly realized by a ReLU network of fixed width and depth linear in the number of iterations, using a universal reflection‑doubling mechanism that replaces two binary transition matrices with a single fixed block matrix and a swap involution. The construction allows exact branch selection via a continuous piecewise‑linear cone switch, propagates full vectorized profiles without decomposing inputs, and handles stage‑dependent forcing while reducing the doubled cascade to a single parity sector through genuine reflection equivariance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
Sep 10

Compressed Recurrent Feedback in Tsetlin Machines: A Reproducible Boolean-FSM Study

The paper proposes a fixed‑width recurrent feedback scheme for Recurrent Tsetlin Machines (RTMs) by folding clause activations with XOR, retaining the folded bits at two time scales, and thresholding them back to binary. This compression reduces 480 clause activations to 96 recurrent bits while achieving comparable accuracy (61.47 ± 6.74% and 62.94 ± 9.92%) on a reproducible Boolean finite‑state‑machine benchmark across 144 runs. The study shows that raw clause feedback offers only marginal accuracy gains but increases recurrent width and execution time significantly, and highlights the importance of no‑feedback controls in sequence model benchmarking.

By Ankit Kumar, Utkarsh Raj, Rishad Shafik, Sudip Roy
arXiv Machine Learning
Aug 27

Gated Recurrent Transformers: Expressive Depth through Recurrent Modulation

The paper introduces Gated Recurrent Transformers, a depth‑sharing architecture that brackets a single shared core with fixed prelude and coda blocks and uses a lightweight projection and element‑wise update gate to modulate recurrent updates. This design allows functional specialization across recurrences while reducing memory footprint. Experiments show that, under equal FLOPs or parameter budgets, the recurrent model matches or surpasses deeper GPT‑2 Small baselines, achieving similar or better accuracy with fewer parameters and lower peak decoding memory.

By Amr Hegazy, Amr Alanwar, Mostafa Elhoushi