← Back to all news
arXiv Computation and Language By Zhenyi Shen, Junru Lu, Wei Liu, Jiangnan Ye, Jiazheng Li, Di Yin, Xing Sun, Tameem Adel, Yali Du, Lin Gui, Yulan He

On the Opportunities and Challenges of Prefill-to-Autoregressive Cross-Model Transfer

Read the original on arXiv Computation and Language →

Prefill-to-Autoregressive Cross-model Transfer (PACT) is a method that lets a large language model provide prompt representations to a smaller model, aiming to combine high capability with efficient generation. The authors present PLatour, an open‑source framework that connects frozen models via lightweight, zero‑initialized linear bridges trained with task rewards and supports hybrid‑attention architectures. While PLatour outperforms open‑source baselines on generation tasks and beats prior results on mathematics and coding benchmarks with a smaller sender, controlled experiments reveal that some gains can arise from uninformative sender signals or receiver‑only training, prompting the authors to propose PACT‑Check to assess genuine communication benefits.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Machine Learning
Jun 4

Breaking the Scale Barrier: One-Shot Knowledge Transfer via Frequency Transform

arXiv:2603. 07523v3 Announce Type: replace Abstract: Transferring knowledge by fine-tuning large-scale pre-trained networks has become a standard paradigm for downstream tasks, yet the knowledge of a pre-trained model is tightly coupled with monolithic architecture, which restricts flexible reuse across models of varying scales.

By Jianlu Shen, Fu Feng, Yucheng Xie, Jiaqi Lv, Xin Geng
fine-tuningbenchmarks
More like this →
arXiv AI
Jun 2

Latent Collaboration in Multi-Agent Systems

arXiv:2511. 20639v3 Announce Type: replace-cross Abstract: Multi-agent systems (MAS) extend large language models (LLMs) from independent single-model reasoning to coordinative system-level intelligence.

By Jiaru Zou, Ruizhong Qiu, Gaotang Li, Xiyuan Yang, Katherine Tieu, Pan Lu, Ke Shen, Hanghang Tong, Yejin Choi, Jingrui He, James Zou, Mengdi Wang, Ling Yang
llmsragagentsdiffusionbenchmarks
More like this →
arXiv Machine Learning
Sep 14

Dead Weights, Live Signals: Feedforward Graphs of Frozen Language Models

The paper introduces a feedforward graph architecture that uses several frozen large language models as computational nodes connected through a shared continuous latent space via learned linear projections. By jointly optimizing projection matrices through backpropagation, the system combines the representations of three small frozen models with two larger ones, culminating in a lightweight cross‑attention output node. With only 17.6 M trainable parameters, the architecture attains state‑of‑the‑art results on ARC‑Challenge, OpenBookQA, and MMLU, surpassing both individual constituent models and parameter‑matched learned classifiers.

By Marcus Armstrong, Navid Ayoobi, Arjun Mukherjee
llmsdiffusionbenchmarks
More like this →