← Back to all news
arXiv Machine Learning By Marcus Armstrong, Navid Ayoobi, Arjun Mukherjee

Dead Weights, Live Signals: Feedforward Graphs of Frozen Language Models

Read the original on arXiv Machine Learning →

The paper introduces a feedforward graph architecture that uses several frozen large language models as computational nodes connected through a shared continuous latent space via learned linear projections. By jointly optimizing projection matrices through backpropagation, the system combines the representations of three small frozen models with two larger ones, culminating in a lightweight cross‑attention output node. With only 17.6 M trainable parameters, the architecture attains state‑of‑the‑art results on ARC‑Challenge, OpenBookQA, and MMLU, surpassing both individual constituent models and parameter‑matched learned classifiers.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Aug 27

Why Does Graph Learning Fail to Fully Benefit from a Text Teacher?

The paper examines a multimodal approach that combines a self‑supervised GNN encoder with an alternating optimization scheme involving a language‑model teacher. Despite the expectation that this joint strategy would enhance predictive performance, the authors find that the combined model fails to deliver significant gains. They identify six key factors—ranging from anchor strength trade‑offs to misaligned representation spaces—that explain why the integration of text knowledge does not fully benefit graph learning.

By Fumiaki Kimino (SOKENDAI), Ryoma Sato (SOKENDAI, National Institute of Informatics)
llmsragmultimodalsafety
More like this →
arXiv Computation and Language
1d ago

On the Opportunities and Challenges of Prefill-to-Autoregressive Cross-Model Transfer

Prefill-to-Autoregressive Cross-model Transfer (PACT) is a method that lets a large language model provide prompt representations to a smaller model, aiming to combine high capability with efficient generation. The authors present PLatour, an open‑source framework that connects frozen models via lightweight, zero‑initialized linear bridges trained with task rewards and supports hybrid‑attention architectures. While PLatour outperforms open‑source baselines on generation tasks and beats prior results on mathematics and coding benchmarks with a smaller sender, controlled experiments reveal that some gains can arise from uninformative sender signals or receiver‑only training, prompting the authors to propose PACT‑Check to assess genuine communication benefits.

By Zhenyi Shen, Junru Lu, Wei Liu, Jiangnan Ye, Jiazheng Li, Di Yin, Xing Sun, Tameem Adel, Yali Du, Lin Gui, Yulan He
llmsbenchmarks
More like this →
arXiv Computation and Language
Oct 1

Learning Functional Subspaces for Neural Network Compression

arXiv:2609.40127v1 Announce Type: cross Abstract: Modern transformers pair impressive capabilities with substantial memory and compute demands. Low-rank weight factorization reduces both while keepin...

By Massimo Bini, Anders Christensen, Stephan Alaniz, Judah Goldfeder, Ole Winther, Yann LeCun, Ravid Shwartz-Ziv, Zeynep Akata
llmsefficiency
More like this →