← Back to all news
arXiv Machine Learning

RAOA: Alternating-Operator Neural Computation with Programmable Radio Propagation

The paper introduces the Radio Alternating Operator Ansatz (RAOA), a recurrent computing architecture that uses programmable radio propagation as a computational depth. RAOA alternates energy‑derived problem updates with mixing updates over a persistent latent state, enabling repeated compositional passes even with the same learned controls. Experiments on discrete optimization, simulation of programmable propagation, and adaptation of pretrained language models show that RAOA can improve solution quality, handle higher‑order interactions, and adapt models with performance close to shallow MLPs, all within a single recurrent framework.

Related stories

arXiv Machine Learning
Aug 27

Gated Recurrent Transformers: Expressive Depth through Recurrent Modulation

The paper introduces Gated Recurrent Transformers, a depth‑sharing architecture that brackets a single shared core with fixed prelude and coda blocks and uses a lightweight projection and element‑wise update gate to modulate recurrent updates. This design allows functional specialization across recurrences while reducing memory footprint. Experiments show that, under equal FLOPs or parameter budgets, the recurrent model matches or surpasses deeper GPT‑2 Small baselines, achieving similar or better accuracy with fewer parameters and lower peak decoding memory.

By Amr Hegazy, Amr Alanwar, Mostafa Elhoushi
llmsefficiency
More like this →
arXiv Machine Learning
Sep 4

Prospective Coding Improves Learning in Deep Continuous-Time Recurrent Networks

The paper introduces Recursive Quadrature Filters (RQFs), complex‑valued temporal filters that act as band‑pass filters within diagonal state‑space models. By making each layer’s bottom‑up input prospective through a parameter‑free two‑tap update, the authors mitigate depth‑dependent gradient attenuation in deep continuous‑time recurrent networks. Experiments on RQFs, S5, and ORGaNICs show that prospective variants match or surpass non‑prospective controls, achieving high accuracy on raw‑audio Speech Commands and the Path‑X task with few parameters.

By Shivang Rawat, Mirko Morello, Flaviano Morone, David J. Heeger
More like this →
arXiv Machine Learning
Aug 4

Structured Recurrent Mixers for Massively Parallelized Sequence Generation

arXiv:2605. 08696v4 Announce Type: replace-cross Abstract: Over the last two decades, language modeling has experienced a shift from the use of predominantly recurrent architectures that process tokens sequentially during training and inference to non-recurrent models that process sequence elements in parallel during training, which results in greater training efficiency and stability at the expense of lower inference throughput.

By Benjamin L. Badger
llmsreinforcement-learningefficiency
More like this →
arXiv AI
Aug 11

Full-bandwidth transformer

arXiv:2608. 08888v1 Announce Type: new Abstract: Autoregressive transformers compute along two axes: horizontally across generated tokens, and vertically through model depth.

By Xi Wang, Ziyang Cai, Zheng Zhan, Harry Dong, Ying Fan, Gustavo de Rosa, Tim Pearce, John Langford
llmsragefficiency
More like this →
arXiv AI
5d ago

Instruction-Conditioned Electromagnetic Spectrum Understanding via Budget-Adaptive Signal Tokenization

The paper introduces BATok, a budget‑adaptive tokenizer that converts raw I/Q electromagnetic signals into compact tokens suitable for multimodal large language models. BATok adjusts token capacity based on input length and signal content, using multi‑resolution features and a local energy prior to resample representations. The authors also present EMSpec‑Instruct, a dataset linking raw signals, waterfall images, and language instructions for tasks such as modulation recognition, structured detection, and signal grounding, and demonstrate that BATok achieves competitive performance across these tasks.

By Lei Zhai, Zhihao Chang, Shuyuan Yang, Zhixi Feng
llmsragnlpmultimodal
More like this →