arXiv AI June 2, 2026 By Xiangyi Chen, Zelun Wang, Xinyi Li, Yi-Ping Hsu, Jaewon Yang, Jiajing Xu

LLMs Need Encoders for Semantic IDs Too

arXiv:2606. 00324v1 Announce Type: cross Abstract: Multimodal LLMs use dedicated encoders to bridge non-language modalities (vision encoders for images, depth models for audio codec tokens) because raw token embeddings alone cannot capture modality-specific structure.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Sep 22

Guiding the coarse levels of semantic IDs makes the fine levels learnable

arXiv:2609.22227v1 Announce Type: cross Abstract: Generative retrieval represents each item by a short Semantic ID and casts recommendation as autoregressive generation of that sequence. Because the...

By Bin Wang, Zhengyu Zhang

llms rag nlp reinforcement-learning safety

arXiv AI

Jul 31

ARC-Encoder: learning compressed text representations for large language models

arXiv:2510. 20535v2 Announce Type: replace-cross Abstract: Recent techniques such as retrieval-augmented generation or chain-of-thought reasoning have led to longer contexts and increased inference costs.

By Hippolyte Pilchen, Edouard Grave, Patrick P\'erez

llms rag fine-tuning benchmarks

arXiv Machine Learning

Aug 3

PaletteID: Prototype-Composed Semantic Identifiers for Multimodal CTR Prediction

arXiv:2607. 29000v1 Announce Type: cross Abstract: Multimodal information can improve the accuracy of click-through rate (CTR) prediction and effectively alleviate item cold-start and long-tail problems.

By Huanyu Liu, Baining Chen, Hui Liu, Zengyang Li, Ziyi Huang

rag multimodal

arXiv AI

Aug 11

Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models

arXiv:2608. 09227v1 Announce Type: new Abstract: Omnimodal language models (OLMs) enable unified audio-visual understanding, but processing long joint token sequences makes inference computationally prohibitive.

By Puneet Mathur, Manan Suri, Dinesh Manocha

llms nlp fine-tuning efficiency multimodal benchmarks

arXiv AI

Jul 29

TopoGR: Revealing and Preserving Latent Structure of Semantic ID in Generative Recommendation

arXiv:2607. 25216v1 Announce Type: cross Abstract: Semantic ID-based generative recommendation tokenizes each item into a sequence of discrete semantic IDs and predicts the next item by generating semantic IDs.

By Ziyu Zheng, Zhengshun Du, Yaming Yang, Bin Tong, Guan Wang, Meng Yan, Ziyu Guan, Wei Zhao

rag nlp benchmarks

arXiv AI

Jul 29

VaLiDRec: Variable-Length LLM-Aligned Semantic IDs for Generative Recommendation

arXiv:2607. 25209v1 Announce Type: cross Abstract: Generative recommendation commonly represents items using fixed-length semantic identifiers (SIDs) constructed through clustering and quantization.

By Shutong Qiao, Wei Yuan, Tong Chen, Hao Wang, Quoc Viet Hung Nguyen, Hongzhi Yin

llms efficiency

One email a morning, machine-written

Related stories

Guiding the coarse levels of semantic IDs makes the fine levels learnable

ARC-Encoder: learning compressed text representations for large language models

PaletteID: Prototype-Composed Semantic Identifiers for Multimodal CTR Prediction

Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models

TopoGR: Revealing and Preserving Latent Structure of Semantic ID in Generative Recommendation

VaLiDRec: Variable-Length LLM-Aligned Semantic IDs for Generative Recommendation