arXiv AI June 2, 2026 By Xutao Ma, Yixiao Huang, Hanlin Zhu, Somayeh Sojoudi

Breaking the Reversal Curse in Autoregressive Language Models via Identity Bridge

arXiv:2602. 02470v2 Announce Type: replace Abstract: Autoregressive large language models (LLMs) have achieved remarkable success in many complex tasks, yet they can still fail in very simple logical reasoning such as the "reversal curse" -- when trained on forward knowledge data of the form "$A \rightarrow B$" (e.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Jul 28

LEDOM: Reverse Language Model

arXiv:2507. 01335v4 Announce Type: replace-cross Abstract: Autoregressive language models are trained exclusively left-to-right.

By Xunjian Yin, Sitao Cheng, Yuxi Xie, Xinyu Hu, Li Lin, Xinyi Wang, Liangming Pan, William Yang Wang, Xiaojun Wan

llms

arXiv AI

Jun 8

Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation

arXiv:2606. 06712v1 Announce Type: cross Abstract: We study the transformation of autoregressive models (ARLMs) into diffusion language models (DLMs).

By Xingyu Su, Jacob Helwig, Shubham Parashar, Atharv Chagi, Lakshmi Jotsna, Degui Zhi, James Caverlee, Dileep Kalathil, Shuiwang Ji

llms diffusion efficiency

arXiv AI

Aug 7

DASH: Divergence-Adaptive Supervision Horizons for On-Policy Self-Distillation of Reasoning Models

arXiv:2608. 06243v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models using automatically verifiable outcome signals, but these signals are typically sparse and at the sequence-level.

By ZhiYan Hou, Xinyu Tang, Hongyan An, Jianjin Zhang, Weizhen Wang, Yunyun Han, Gengsheng Li, Xiangzhao Hao, Haiyun Guo, Wenbin Hu, Jinqiao Wang, Yafeng Deng

llms reinforcement-learning efficiency benchmarks

arXiv AI

Jul 10

ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation

arXiv:2502. 15543v4 Announce Type: replace-cross Abstract: Large language models (LLMs) integrated with retrieval-augmented generation (RAG) have improved factuality by grounding outputs in external evidence.

By Pengcheng Huang, Zhenghao Liu, Yukun Yan, Haiyan Zhao, Xiaoyuan Yi, Hao Chen, Zhiyuan Liu, Maosong Sun, Tong Xiao, Ge Yu, Chenyan Xiong

llms rag benchmarks

arXiv Machine Learning

1d ago

MoRFI: Monotonic Sparse Autoencoder Feature Identification

arXiv:2604. 26866v2 Announce Type: replace-cross Abstract: Large language models (LLMs) acquire most of their factual knowledge during the pre-training stage, through next token prediction.

By Dimitris Dimakopoulos, Shay B. Cohen, Ioannis Konstas

llms fine-tuning safety

arXiv Machine Learning

Jun 11

Counterexample Guided Learning in the Large using Reasoning Agents

arXiv:2606. 11521v1 Announce Type: new Abstract: LLMs and LLM agents should improve when given feedback, but identifying when they are able to do so is difficult: feedback is heterogeneous, domain-specific, and difficult to control.

By Hongyi Liu, Frederic Sala, Thomas Reps, Adithya Murali

llms agents

Related stories

LEDOM: Reverse Language Model

Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation

DASH: Divergence-Adaptive Supervision Horizons for On-Policy Self-Distillation of Reasoning Models

ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation

MoRFI: Monotonic Sparse Autoencoder Feature Identification

Counterexample Guided Learning in the Large using Reasoning Agents