arXiv Computation and Language August 31, 2026 By Amit Dhurandhar, Vijil Chenthamarakshan, Dennis Wei, Tejaswini Pedapati, Karthikeyan Natesan Ramamurthy, Rahul Nair

CoFrGeNet: Continued Fraction Architectures for Language Generation

Read the original on arXiv Computation and Language →

CoFrGeNet introduces Continued Fraction Generative Networks, a new function class that replaces Multi-head Attention and Feed-Forward Networks in Transformer blocks with fewer parameters. The architecture includes custom gradient formulations for efficient optimization and can be plugged into existing Transformer workflows with minimal changes. Experiments on GPT2‑xl and Llama3 show competitive or superior performance on downstream tasks while using 1/2 to 2/3 of the original parameters and shorter pre‑training time.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

llms

arXiv AI

Aug 11

Full-bandwidth transformer

arXiv:2608. 08888v1 Announce Type: new Abstract: Autoregressive transformers compute along two axes: horizontally across generated tokens, and vertically through model depth.

By Xi Wang, Ziyang Cai, Zheng Zhan, Harry Dong, Ying Fan, Gustavo de Rosa, Tim Pearce, John Langford

llms rag efficiency

arXiv Machine Learning

Jun 3

Dynamic Short Convolutions Improve Transformers

arXiv:2606. 03825v1 Announce Type: new Abstract: Transformers have become the dominant architecture for large language models, largely due to the scalability and flexibility of attention, feed-forward layers, residual connections, and normalization.

By Oliver Sieberling, Bharat Runwal, Rameswar Panda, Yoon Kim

llms safety

arXiv AI

Aug 5

On the Limits of Layer Pruning for Generative Reasoning in Large Language Models

arXiv:2602. 01997v3 Announce Type: replace-cross Abstract: Recent work has shown that layer pruning can effectively compress large language models (LLMs) while retaining strong performance on classification benchmarks, often with little or no finetuning.

By Safal Shrestha, Anubhav Shrestha, Minwu Kim, Aadim Nepal, Keith Ross

llms fine-tuning efficiency benchmarks

arXiv AI

Jun 16

Long-Context Modeling via GSS-Transformer Hybrid Architecture with Learnable Mixing

arXiv:2606. 16093v1 Announce Type: cross Abstract: Modeling long-range dependencies remains a central challenge in natural language processing.

By Kuzey Torlak, H\"useyin Arda Arslan, An{\i}l Dervi\c{s}o\u{g}lu, Beyza Nur Deniz, Onur Boyar

llms nlp efficiency

arXiv Machine Learning

Aug 19

Recirculation

The paper introduces recirculation, an inference‑time architectural enhancement for foundation models that reduces perplexity and improves accuracy on generation and reasoning tasks without adding significant latency. Recirculation adds a specific form of recurrence, enabling the model to function as a dynamical system that tracks belief states, and is distinct from chain‑of‑thought or depth‑recurrence methods. An adaptive variant requires minimal hyperparameter tuning and achieves notable gains on the Gemma3 family, including a 23% perplexity drop and a 21% accuracy increase on GSM8k.

By Michael C. Mozer, Shoaib Ahmed Siddiqui, Danny Sawyer, Sunny Sanyal, Rosanne Liu

llms

arXiv AI

Jul 17

T^2MLR: Transformer with Temporal Middle-Layer Recurrence

arXiv:2607. 15178v1 Announce Type: cross Abstract: Transformer reasoning is limited by autoregressive decoding, which repeat edly compresses rich hidden computation through token space and makes it difficult for intermediate reasoning states to persist across time.

By Ziyang Cai, Xingyu Zhu, Yihe Dong, Yinghui He, Sanjeev Arora

llms fine-tuning

One email a morning, machine-written

Related stories

Full-bandwidth transformer

Dynamic Short Convolutions Improve Transformers

On the Limits of Layer Pruning for Generative Reasoning in Large Language Models

Long-Context Modeling via GSS-Transformer Hybrid Architecture with Learnable Mixing

Recirculation

T^2MLR: Transformer with Temporal Middle-Layer Recurrence