Prefill-to-Autoregressive Cross-model Transfer (PACT) is a method that lets a large language model provide prompt representations to a smaller model, aiming to combine high capability with efficient generation. The authors present PLatour, an open‑source framework that connects frozen models via lightweight, zero‑initialized linear bridges trained with task rewards and supports hybrid‑attention architectures. While PLatour outperforms open‑source baselines on generation tasks and beats prior results on mathematics and coding benchmarks with a smaller sender, controlled experiments reveal that some gains can arise from uninformative sender signals or receiver‑only training, prompting the authors to propose PACT‑Check to assess genuine communication benefits.
arXiv:2610.19696v1 Announce Type: new Abstract: Multi-agent systems allow language models to solve tasks collaboratively by sharing complementary information. These systems typically communicate thro...
arXiv:2603. 07523v3 Announce Type: replace Abstract: Transferring knowledge by fine-tuning large-scale pre-trained networks has become a standard paradigm for downstream tasks, yet the knowledge of a pre-trained model is tightly coupled with monolithic architecture, which restricts flexible reuse across models of varying scales.
arXiv:2511. 20639v3 Announce Type: replace-cross Abstract: Multi-agent systems (MAS) extend large language models (LLMs) from independent single-model reasoning to coordinative system-level intelligence.
The paper introduces a feedforward graph architecture that uses several frozen large language models as computational nodes connected through a shared continuous latent space via learned linear projections. By jointly optimizing projection matrices through backpropagation, the system combines the representations of three small frozen models with two larger ones, culminating in a lightweight cross‑attention output node. With only 17.6 M trainable parameters, the architecture attains state‑of‑the‑art results on ARC‑Challenge, OpenBookQA, and MMLU, surpassing both individual constituent models and parameter‑matched learned classifiers.
arXiv:2608. 11676v1 Announce Type: new Abstract: Heterogeneous multi-agent LLM systems, where agents are powered by different model families, can outperform homogeneous configurations by reducing redundant reasoning patterns.
arXiv:2609.38149v1 Announce Type: new Abstract: Transformer language models (LMs) are feed-forward: deep-layer representations are never fed back to shallower layers, and the only pathway for informa...
arXiv:2603.18908v5 Announce Type: replace Abstract: Independently trained language models often learn compatible late-stage representations, despite differences in training objectives, architectures,...
Circe is a training‑free method for self‑evaluating large language model responses without reference answers. It combines internal signals—component interaction consistency (CIC) and representation geometry (RG)—with an external likelihood confidence (LC) score to produce a unified correctness score. Experiments on five LLMs across six benchmarks demonstrate that Circe better distinguishes correct from incorrect responses than existing baselines and works in multilingual, multimodal, and open‑world settings.
arXiv:2511. 05313v2 Announce Type: replace Abstract: The substantial inference costs of attention in transformers motivated the development of efficient sequence mixers: namely sparse and sliding window attention, convolutions and linear attention.
GEPARD is a streaming text‑to‑speech model that uses a standard large language model backbone to generate speech autoregressively, decoding audio with an FSQ‑based neural codec. It streams audio chunk‑by‑chunk as text arrives, achieving a real‑time factor of about 0.067 and an aggregate speedup of roughly 204× on a single GPU with 256 concurrent streams. The design keeps all complex auxiliary mechanisms outside the decode loop, enabling deployment with a standard LLM engine (vLLM) without kernel modifications.
The paper introduces Local Prototype Reconstruction (LPR), a lightweight regularizer that encourages bridge embeddings in speech‑to‑LLM systems to stay close to the frozen LLM’s input‑embedding neighbourhoods. By measuring lexical manifold compatibility with a fixed, head‑free, timestamp‑free diagnostic, the authors show that next‑word prediction and sentence‑level contrastive pretraining do not fully capture token‑level lexical compatibility. Experiments on multilingual ASR and speech translation demonstrate that LPR improves transfer, especially for translation and low‑resource adaptation, and that the diagnostic correlates with downstream gains across objectives.
arXiv:2606. 28182v1 Announce Type: cross Abstract: Embodied agents operating in decentralized and partially observable environments have attracted growing attention in recent years.