arXiv Machine Learning June 2, 2026 By Bencheng Yan, Yuejie Lei, Zhiyuan Zeng, Zheye Deng, Di Wang, Kaiyi Lin, Pengjie Wang, Chuan Yu, Jian Xu, Bo Zheng

From Scaling to Structured Expressivity: Rethinking Transformers for CTR Prediction

Read the original on arXiv Machine Learning →

arXiv:2511. 12081v2 Announce Type: replace-cross Abstract: Despite massive investments in scale, deep models for click-through rate (CTR) prediction often exhibit rapidly diminishing returns -- a stark contrast to the {predictable scaling laws} seen in large language models (LLMs).

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 28

MMOE: Modernizing Diffusion Transformers with Efficient Expert Design

arXiv:2607. 24665v1 Announce Type: cross Abstract: Modern large language models scale successfully by pairing capacity growth with efficiency, keeping per-token and deployment costs under control as capacity grows.

By Yanhao Jia, Jiepeng Wang, Haibin Huang, Chi Zhang, Erik Cambria, Xuelong Li

llms diffusion efficiency

arXiv AI

1d ago

SAPE: Sandwich Adapters for Parameter Efficiency in Large Language Model Fine-Tuning

arXiv:2608. 15360v1 Announce Type: cross Abstract: While Parameter-Efficient Fine-Tuning (PEFT) has substantially reduced the hardware cost of adapting Large Language Models (LLMs) by decreasing the number of trainable parameters, recent studies have sought to further improve PEFT through parameter sharing.

By Mohammad Aref Jafari-Raddani, Morteza Mohajjel Kafshdooz

llms rag fine-tuning benchmarks

Hugging Face Trending Papers

Jul 9

Hidden Decoding at Scale: Latent Computation Scaling for Large Language Models

Scaling Large Language Models (LLMs) has been driven mainly by enlarging the Transformer backbone, but for an already-strong model this requires another round of costly pretraining. We study whether an existing backbone can keep improving by allocating more computation to each token while leaving the Transformer backbone fixed.

llms rag

arXiv Machine Learning

Jul 7

Equivalence of Context and Parameter Updates in Modern Transformer Blocks

arXiv:2511. 17864v3 Announce Type: replace Abstract: Recent research has established that the impact of context in a vanilla transformer can be represented implicitly by forming a token-dependent, rank-1 patch to its MLP weights.

By Adrian Goldwaser, Michael Munn, Javier Gonzalvo, Benoit Dherin

llms

arXiv AI

Jul 29

At-the-Roofline Sparse Tensor Contractions on Vector Processors for Transformer Inference

arXiv:2607. 25504v1 Announce Type: cross Abstract: Fine-grained weight pruning and activation sparsification have emerged as effective approaches for reducing the compute and memory cost of inference for Transformer models.

By Bowen Wang, Chi Zhang, Diyou Shen, Renzo Andri, Navaneeth Kunhi Purayil, Luca Benini

llms efficiency

arXiv AI

Jul 14

Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models

arXiv:2601. 07372v2 Announce Type: replace-cross Abstract: While Mixture-of-Experts (MoE) scales capacity via conditional computation, Transformers lack a native primitive for knowledge lookup, forcing them to inefficiently simulate retrieval through computation.

By Xin Cheng, Rui Tian, Wangding Zeng, Damai Dai, Qinyu Chen, Bingxuan Wang, Zhenda Xie, Kezhao Huang, Xingkai Yu, Chengqi Deng, Shangyan Zhou, Chenggang Zhao, Zhewen Hao, Yukun Li, Han Zhang, Zhengyan Zhang, Yixu Wei, M. Y Xu, Huishuai Zhang, Dongyan Zhao, Wenfeng Liang

llms rag efficiency benchmarks

Related stories

MMOE: Modernizing Diffusion Transformers with Efficient Expert Design

SAPE: Sandwich Adapters for Parameter Efficiency in Large Language Model Fine-Tuning

Hidden Decoding at Scale: Latent Computation Scaling for Large Language Models

Equivalence of Context and Parameter Updates in Modern Transformer Blocks

At-the-Roofline Sparse Tensor Contractions on Vector Processors for Transformer Inference

Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models