← Back to all news
arXiv Machine Learning By Lecheng Kong, Xiyuan Wang, Yixin Chen, Muhan Zhang

Round-trip Reinforcement Learning: Self-Consistent Training for Better Chemical LLMs

Read the original on arXiv Machine Learning →

The paper introduces Round-Trip Reinforcement Learning (RTRL), a framework that trains chemical language models to improve round‑trip consistency by rewarding successful forward and reverse transformations. By iteratively training forward and reverse mappings, RTRL leverages abundant unlabeled chemical data to enhance both consistency and overall performance across supervised, self‑supervised, and synthetic data regimes. Experiments show that RTRL outperforms strong baselines, demonstrating that round‑trip consistency can be treated as a trainable objective for more robust foundation models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Aug 20

Training Chemical Plausibility-Aware Large Language Models for Single-Step Retrosynthesis

The paper introduces Top‑K prompting as a training and inference strategy to better capture the diverse, plausible predictions inherent in single‑step retrosynthesis. Using an ultra‑large dataset (CREED‑CCV‑2+USPTO‑XL) of ~45.6 million verified reactions, the authors train the Chemistry Constraint‑Consistent Language Model (C3LM). With fine‑tuning that incorporates ChemCensor‑based and novelty‑oriented rewards, C3LM achieves state‑of‑the‑art performance on the OOD URSA‑expert‑2026 benchmark and demonstrates complementary reaction space exploration compared to conventional models, suggesting benefits for ensemble‑based retrosynthesis systems.

By Bogdan Zagribelnyy, Ivan Ilin, Nikita Bondarev, Maksim Kuznetsov, Mathieu Reymond, Vladimir Aladinskiy, Alex Aliper, Alex Zhavoronkov
llmsfine-tuningbenchmarks
More like this →
arXiv AI
Aug 20

PGFS++: Molecular Property Improvement under Synthesis and Diversity Constraints

PGFS++ is a synthesis‑aware reinforcement learning framework that improves molecular properties while ensuring the resulting molecules can be synthesized and remain structurally similar to the input. It builds on PGFS+ by using trainable embedding lookup tables for reaction templates and second reactants, a more effective scoring function, and a refined RL algorithm. Experiments demonstrate that PGFS++ enhances target properties and preserves high output diversity, overcoming the reward‑hacking failure mode seen in earlier versions.

By Boqiao Zhang, Godbless James, Sai Krishna Gottipati, Andrew Fitzgibbon
ragreinforcement-learning
More like this →
Hugging Face Trending Papers
Aug 19

PGFS++: Molecular Property Improvement under Synthesis and Diversity Constraints

PGFS++ is a synthesis‑aware reinforcement learning framework that improves molecular properties such as drug‑likeness or binding affinity while ensuring the resulting molecules can be synthesized and remain structurally similar to the input. It builds on PGFS+ by using trainable embedding lookup tables for reaction templates and second reactants, a more effective scoring function, and a refined RL algorithm. The method addresses a reward‑hacking failure mode by treating each input molecule as the start of a forward‑synthesis trajectory, applying learned reaction templates with in‑stock building blocks, and producing diverse, high‑quality outputs with explicit synthesis routes.

ragreinforcement-learning
More like this →