arXiv Machine Learning By Wuyang Zhou, Yuxuan Gu, Giorgos Iacovides, Yuning Qiu, Qibin Zhao, Danilo Mandic

Tensorizing Engram: Sharing Latents Across N-Gram Embeddings is Beneficial in LLMs

Read the original on arXiv Machine Learning →

arXiv:2606. 08347v1 Announce Type: cross Abstract: Modern language models represent text using discrete token-level embeddings, which forces recurring multi-token patterns to be learned implicitly across Transformer layers.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI
Jun 2

LLMs Need Encoders for Semantic IDs Too

arXiv:2606. 00324v1 Announce Type: cross Abstract: Multimodal LLMs use dedicated encoders to bridge non-language modalities (vision encoders for images, depth models for audio codec tokens) because raw token embeddings alone cannot capture modality-specific structure.

By Xiangyi Chen, Zelun Wang, Xinyi Li, Yi-Ping Hsu, Jaewon Yang, Jiajing Xu