arXiv Computation and Language November 4, 2026 By Abdellah Rahmani, Sean McLeish, Ang Li, Sanae Lotfi, Pascal Frossard, Micah Goldblum

Library of Layers (LoL): Adaptively Ordering Transformer Layers at Runtime

Read the original on arXiv Computation and Language →

The paper introduces Library of Layers (LoL), an architecture that dynamically orders transformer layers during inference, allowing a single model to adapt its computation to the task at hand. LoL’s single hyperparameter—the size of its global library of transformer blocks—enables a smooth transition between parameter reuse and sparsity, unifying dense, depth‑recurrent, and mixture‑of‑experts trade‑offs. Experiments show LoL matches or exceeds these baselines on text, synthetic Sudoku and maze tasks, and demonstrates task‑specific routing in multilingual and safety‑focused settings.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI

Aug 18

SAPE: Sandwich Adapters for Parameter Efficiency in Large Language Model Fine-Tuning

arXiv:2608. 15360v1 Announce Type: cross Abstract: While Parameter-Efficient Fine-Tuning (PEFT) has substantially reduced the hardware cost of adapting Large Language Models (LLMs) by decreasing the number of trainable parameters, recent studies have sought to further improve PEFT through parameter sharing.

By Mohammad Aref Jafari-Raddani, Morteza Mohajjel Kafshdooz

llms rag fine-tuning benchmarks

arXiv AI

Jun 26

Tuning Language Models by Mixture-of-Depths Ensemble

arXiv:2410. 13077v2 Announce Type: replace-cross Abstract: Transformer-based Large Language Models (LLMs) traditionally rely on final-layer loss for finetuning and final-layer representations for predictions, potentially overlooking the predictive power embedded in late layers.

By Haoyan Luo, Lucia Specia

llms fine-tuning safety

arXiv AI

Jul 14

Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models

arXiv:2601. 07372v2 Announce Type: replace-cross Abstract: While Mixture-of-Experts (MoE) scales capacity via conditional computation, Transformers lack a native primitive for knowledge lookup, forcing them to inefficiently simulate retrieval through computation.

By Xin Cheng, Rui Tian, Wangding Zeng, Damai Dai, Qinyu Chen, Bingxuan Wang, Zhenda Xie, Kezhao Huang, Xingkai Yu, Chengqi Deng, Shangyan Zhou, Chenggang Zhao, Zhewen Hao, Yukun Li, Han Zhang, Zhengyan Zhang, Yixu Wei, M. Y Xu, Huishuai Zhang, Dongyan Zhao, Wenfeng Liang

llms rag efficiency benchmarks

arXiv AI

Sep 16

Thinking Deeper, Not Longer: Memory-Efficient Test-Time Reasoning with Depth-Recurrent Transformers for Compositional Generalization

arXiv:2603.21676v2 Announce Type: replace-cross Abstract: Standard Transformers have a fixed computational depth, limiting their ability to generalize to tasks that require variable-depth reasoning....

By Hung-Hsuan Chen

llms safety

arXiv Computation and Language

Oct 21

From Abstraction to Action: DSLs for Natural Language Abstraction Interpretation and Execution

The paper introduces domain-specific languages (DSLs) as an explicit representation layer to bridge the gap between high-level natural-language abstractions and concrete actions. Experiments on the visually-grounded “Hexagons” task and the “Text2CAD” benchmark show that code generation using a human-curated DSL yields dramatic performance gains, especially at higher abstraction levels, while automatically induced DSLs only partially recover these gains. The study highlights that the right representation can close much of the performance gap caused by abstraction, but that automatically learning such representations remains a challenge.

By Amit Lisha, Reut Tsarfaty

llms agents benchmarks

arXiv AI

Oct 1

What Limits Recursive Reasoning Models: Optimization, Architecture and Test-Time Scaling

arXiv:2609.39967v1 Announce Type: cross Abstract: Recursive reasoning models apply a small shared Transformer block many times to refine a latent state. This gives them large effective depth with few...

By Yuliana Shakhvalieva, Dmitrii Kharchev, Viacheslav Bezrukov, Inessa Fedorova, Dmitry Bocharov, Ivan Oseledets, Valerii Ternovskii

llms

One email a morning, machine-written

Related stories

SAPE: Sandwich Adapters for Parameter Efficiency in Large Language Model Fine-Tuning

Tuning Language Models by Mixture-of-Depths Ensemble

Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models

Thinking Deeper, Not Longer: Memory-Efficient Test-Time Reasoning with Depth-Recurrent Transformers for Compositional Generalization

From Abstraction to Action: DSLs for Natural Language Abstraction Interpretation and Execution

What Limits Recursive Reasoning Models: Optimization, Architecture and Test-Time Scaling