arXiv AI

When Machines Speak: A Unified Generative Framework for Integrating Machine-Native Symbols into Pretrained Large Language Models

arXiv:2608. 19529v1 Announce Type: cross Abstract: Many real-world AI systems represent entities, behaviors, and structured information using discrete machine-native symbols rather than natural language.

arXiv AI

2d ago

Grounded Token Initialization for New Vocabulary in LMs for Generative Recommendation

arXiv:2604.02324v2 Announce Type: replace-cross Abstract: Language models (LMs) are increasingly extended with new learnable vocabulary tokens for domain-specific tasks, such as Semantic-ID tokens in...

By Daiwei Chen, Zhoutong Fu, Chengming Jiang, Haichao Zhang, Ran Zhou, Tan Wang, Chunnan Yao, Guoyao Li, Rui Cai, Yihan Cao, Ruijie Jiang, Fedor Borisyuk, Jianqiang Shen, Jingwei Wu, Ramya Korlakai Vinayak

llms rag fine-tuning benchmarks

arXiv Machine Learning

Jun 10

Generative Archetype-Grounded Item Representations for Sequential Recommendation

arXiv:2606. 11023v1 Announce Type: cross Abstract: Sequential recommendation aims to predict users' next interaction with items by analyzing their historical behavior.

By Yifan Li, Jiahong Liu, Xinni Zhang, Hao Chen, Yankai Chen, Wenhao Yu, Jianting Chen, Irwin King

llms rag benchmarks

Hugging Face Trending Papers

Jun 9

Generative Archetype-Grounded Item Representations for Sequential Recommendation

Sequential recommendation aims to predict users' next interaction with items by analyzing their historical behavior. However, the limited quality of item representations remains a critical bottleneck.

llms rag benchmarks

arXiv Machine Learning

Jun 4

Can Large Language Models Generalize Procedures Across Representations?

arXiv:2602. 03542v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are trained and tested extensively on symbolic representations such as code and graphs, yet real-world user tasks are often specified in natural language.

By Fangru Lin, Valentin Hofmann, Xingchen Wan, Weixing Wang, Zifeng Ding, Anthony G. Cohn, Janet B. Pierrehumbert

llms reinforcement-learning

arXiv AI

Jun 3

ReaLM: Residual Quantization Bridging Knowledge Graph Embeddings and Large Language Models

arXiv:2510. 09711v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have recently emerged as a powerful paradigm for Knowledge Graph Completion (KGC), offering strong reasoning and generalization capabilities beyond traditional embedding-based approaches.

By Wenbin Guo, Xin Wang, Jiaoyan Chen, Lingbing Guo, Zhao Li, Zirui Chen

llms rag nlp efficiency benchmarks

arXiv Machine Learning

2d ago

Squeezing More from Limited Data with Recursive Transformers

The paper investigates how to effectively pre‑train language models when the data budget is limited but compute is plentiful. It shows that increasing model size only improves performance up to an optimal point, after which overfitting degrades generalization, and that this optimal size varies with both the data budget and downstream tasks. To overcome the inefficiencies of standard Transformers in this regime, the authors propose recursive Transformers that reuse a shared block across depth and employ factorized embeddings, achieving better results than standard models on 10M–100M word pre‑training budgets and competitive performance with BabyLM Challenge 2025 winners.

By Serdar G\"ulbahar, Lukas Edman, Alexander Fraser

llms rag

arXiv AI

Aug 12

JEPA-DNA: Grounding Genomic Foundation Models through Joint-Embedding Predictive Architectures

arXiv:2602. 17162v3 Announce Type: replace Abstract: Genomic Foundation Models (GFMs) typically rely on Masked Language Modeling (MLM) or Next-Token Prediction (NTP) to learn the "Laws of Nature".

By Ariel Larey, Elay Dahan, Amit Bleiweiss, Raizy Kellerman, Guy Leib, Omri Nayshool, Dan Ofer, Tal Zinger, Dan Dominissini, Gideon Rechavi, Nicole Bussola, Simon Lee, Shane O'Connell, Dung Hoang, Marissa Wirth, Alexander W. Charney, Nati Daniel, Yoli Shavit

rag diffusion benchmarks safety

arXiv AI

Jun 18

From Values to Tokens: An LLM-Driven Framework for Context-aware Time Series Forecasting via Symbolic Discretization

arXiv:2508. 09191v2 Announce Type: replace-cross Abstract: Time series forecasting plays a vital role in supporting decision-making across a wide range of critical applications, including energy, healthcare, and finance.

By Xiaoyu Tao, Shilong Zhang, Mingyue Cheng, Daoyu Wang, Tingyue Pan, Bokai Pan, Changqing Zhang, Shijin Wang

llms nlp fine-tuning safety

arXiv Machine Learning

Jul 14

CoGenCast: A Coupled Autoregressive-Flow Generative Framework for Time Series Forecasting

arXiv:2602. 03564v2 Announce Type: replace Abstract: Time series forecasting can be viewed as a generative problem that requires both semantic understanding over contextual conditions and stochastic modeling of continuous temporal dynamics.

By Mingyue Cheng, Yaguo Liu, Daoyu Wang, Xiaoyu Tao, Qi Liu

llms diffusion multimodal benchmarks

arXiv Machine Learning

Jun 16

Next-Latent Prediction Transformers Learn Compact World Models

arXiv:2511. 05963v4 Announce Type: replace Abstract: Transformers replace recurrence with a memory that grows with sequence length and self-attention that enables ad-hoc lookups over past tokens.

By Jayden Teoh, Manan Tomar, Kwangjun Ahn, Edward S. Hu, Tim Pearce, Pratyusha Sharma, Akshay Krishnamurthy, Riashat Islam, Alex Lamb, John Langford

llms diffusion benchmarks safety

arXiv Machine Learning

Jul 21

Token-Level Off-Policy Learning for Faithful Generation Under Distribution Shift

arXiv:2607. 17524v1 Announce Type: cross Abstract: We propose Token-Level Off-Policy Labeling (TOPL), an off-policy training paradigm that reframes post-training as a token-level correctness prediction task.

By Zitong Huang, Gustavo Lucas Carvalho, Deqing Fu, Robin Jia

nlp fine-tuning benchmarks

arXiv Machine Learning

Aug 11

Preserving Item Semantics for Free: Rethinking Token Initialization in LLM-Based Generative Recommendation

arXiv:2608. 07816v1 Announce Type: cross Abstract: Recent advances in generative recommendation (GR) leverage large language models (LLMs) as recommender backbones, enabling LLMs to directly generate recommendations conditioned on item-interaction histories.

By Donald Loveland, Liam Collins, Bhuvesh Kumar, Danai Koutra, Neil Shah

llms rag fine-tuning

Related stories