← Back to all news
arXiv Machine Learning By Siavash Golkar, Jake Kovalic, Shirley Ho

AI Mindmeld: A Latent Interconnect Protocol

Read the original on arXiv Machine Learning →

The paper introduces the Latent Interconnect Protocol (LIP), a framework that allows large language models (LLMs) to directly exchange rich, high‑dimensional representations with specialist models (e.g., protein and small‑molecule experts) via learned connectors. By training only these connectors, the approach enables zero‑shot transfer to cross‑domain tasks and compositional reasoning without joint training of the LLMs or specialists. Experiments with Gemma 3 and Kimi K2.6 show improved task performance and compositionality, especially in zero‑shot settings, while highlighting the need for consistent specialist interfaces and LLM training to handle such inputs.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Computer Vision
Sep 14

Retrieved Images as Visual Thought: Training-Free Multimodal In-Context Learning for the Open-vs-Closed Gap

ReVisIT is a train‑free framework that turns retrieved image‑label pairs into units of visual thought, combining structured class definitions, multimodal retrieval, and alternating user/assistant injection before joint decoding. On several benchmarks—including Fast Open MiniImageNet, Bongard‑OpenWorld, and the newly released MAAC‑Bench—ReVisIT achieves performance comparable to or surpassing large, trained models while using far fewer parameters. The approach demonstrates that high‑quality retrieval and a simple turns layer can provide a universal performance boost across diverse multimodal tasks.

By Bingchen Huang, Zhiling Wang, Yifu Chen, Yuanchao Du
llmsmultimodalbenchmarks
More like this →
arXiv AI
6d ago

SkillMem: Compact External Skill Memory for Latent Multi-Agent Reasoning

SkillMem introduces a compact external latent memory for reusable skills in latent multi‑agent systems, enabling role‑conditioned cross‑attention to guide agents without storing large per‑skill parameters. Trained via behavioral distillation, SkillMem matches the effect of textual skills while keeping the backbone frozen. Across six benchmarks and Qwen3 models from 4B to 14B, it delivers strong gains for smaller backbones, reduces skill storage by over 38×, and achieves up to 4.23× inference speedup.

By Yuxuan Zhu, Mohammad Mohammadi Amiri, Pin-Yu Chen, Keerthiram Murugesan
ragagentsefficiencybenchmarks
More like this →
arXiv Computation and Language
Sep 28

Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forward

The paper introduces UniSandbox, a decoupled evaluation framework with controlled synthetic datasets, to study whether understanding informs generation in Unified Multimodal Models. Results show a notable understanding‑generation gap, especially in reasoning generation and knowledge transfer. Explicit Chain‑of‑Thought (CoT) in the understanding module bridges this gap, and self‑training can internalize CoT for implicit reasoning during generation; query‑based architectures also exhibit latent CoT‑like properties that aid knowledge transfer.

By Yuwei Niu, Weiyang Jin, Jiaqi Liao, Chaoran Feng, Peng Jin, Bin Lin, Zongjian Li, Bin Zhu, Weihao Yu, Li Yuan
multimodal
More like this →