The paper examines a multimodal approach that combines a self‑supervised GNN encoder with an alternating optimization scheme involving a language‑model teacher. Despite the expectation that this joint strategy would enhance predictive performance, the authors find that the combined model fails to deliver significant gains. They identify six key factors—ranging from anchor strength trade‑offs to misaligned representation spaces—that explain why the integration of text knowledge does not fully benefit graph learning.
By Fumiaki Kimino (SOKENDAI), Ryoma Sato (SOKENDAI, National Institute of Informatics)
arXiv:2605.10247v2 Announce Type: replace
Abstract: Applying Large Language Models (LLMs) to graph-structured data usually involves multi-step pipelines in which textual node attributes are compresse...
By Dario Vajda
Prefill-to-Autoregressive Cross-model Transfer (PACT) is a method that lets a large language model provide prompt representations to a smaller model, aiming to combine high capability with efficient generation. The authors present PLatour, an open‑source framework that connects frozen models via lightweight, zero‑initialized linear bridges trained with task rewards and supports hybrid‑attention architectures. While PLatour outperforms open‑source baselines on generation tasks and beats prior results on mathematics and coding benchmarks with a smaller sender, controlled experiments reveal that some gains can arise from uninformative sender signals or receiver‑only training, prompting the authors to propose PACT‑Check to assess genuine communication benefits.
By Zhenyi Shen, Junru Lu, Wei Liu, Jiangnan Ye, Jiazheng Li, Di Yin, Xing Sun, Tameem Adel, Yali Du, Lin Gui, Yulan He
arXiv:2603.18908v5 Announce Type: replace
Abstract: Independently trained language models often learn compatible late-stage representations, despite differences in training objectives, architectures,...
By Matt Gorbett, Suman Jana
arXiv:2609.38149v1 Announce Type: new
Abstract: Transformer language models (LMs) are feed-forward: deep-layer representations are never fed back to shallower layers, and the only pathway for informa...
By Dor Tirosh, Ido Amos, Mor Geva
arXiv:2609.40127v1 Announce Type: cross
Abstract: Modern transformers pair impressive capabilities with substantial memory and compute demands. Low-rank weight factorization reduces both while keepin...
By Massimo Bini, Anders Christensen, Stephan Alaniz, Judah Goldfeder, Ole Winther, Yann LeCun, Ravid Shwartz-Ziv, Zeynep Akata
arXiv:2607. 16637v1 Announce Type: new Abstract: Full fine-tuning remains a strong way to adapt pretrained LLMs, but it updates all weights and can be expensive.
By Abdulkadir Erol, Yash Mahajan, Vepaul Hariprashad, Baha Rababah, Santu Karmaker, Cuneyt G. Akcora, Mubarak Shah
arXiv:2607. 00510v1 Announce Type: new Abstract: Knowing which training examples drive outputs is fundamental to auditing, correcting, and understanding language models, yet for modern LLMs this remains expensive, approximate, and largely post-hoc.
By Dan Ley, Giang Nguyen, Himabindu Lakkaraju, Julius Adebayo
The paper introduces Untied Self-Conditioning, a sampler that corrects a train–inference mismatch in flow‑matching language models. By dampening redundant directions in the self‑conditioning input and approximating a step‑average prediction from history, the method improves generation quality without retraining. On LangFlow and ELF‑B datasets, it dramatically lowers perplexity and is preferred in the majority of pairwise comparisons.
By Bocheng Li, Linli Xu
arXiv:2608. 07851v1 Announce Type: new Abstract: Residual connections rely on a static residual pathway, and are essential for training deep neural networks.
By Yuxuan Gu, Wuyang Zhou, Huijun Xing, Danilo Mandic
The paper introduces NinaXander, a method for composing frozen language models from different architecture families by inserting a trained shared‑latent adapter between their layers. By running the initial layers of one model, converting the intermediate representation with the adapter, and then continuing with the remaining layers of another model, multiple composed models can be created without retraining. Experiments with RWKV and Pythia show that while some compositions preserve syntactic quality and reduce memory usage, none match the parent model’s accuracy and language‑modeling performance drops on out‑of‑domain data.
By Takanori Kotama, Shun-ichiro Hayashi, Daichi Mukunoki, Tetsuya Hoshino, Takahiro Katagiri
arXiv:2609.15982v1 Announce Type: cross
Abstract: Skills extend an LLM agent beyond its parametric knowledge, and the gain they promise rests on picking the right one. Deployed harnesses route by pre...
By Ruishuo Chen, Xun Wang, Yu Chen, Zhuoran Li, Longbo Huang