Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

6,934 stories · RSS feed

arXiv AI
Sep 11

Smart Adaptive Computing Across the Continuum: LLMs in IoT-Edge-Cloud Resource Management

The paper introduces a taxonomy for resource management systems that span IoT, edge, and cloud layers, extending Wang et al.’s Continuum Orchestration Systems with two new dimensions: the AI Augmentation Paradigm and the Feedback channel. It evaluates six recent architectures and finds that none combine full LLM orchestration with full agent‑layer feedback in a Cloud Continuum setting, highlighting a gap in cross‑tier feedback abstraction. The study emphasizes the need for a unified feedback mechanism to bridge disparate per‑tier signals to the LLM orchestrator.

By Antonino Vaccarella, Lanpei Li, Vincenzo Lomonaco, Massimo Coppola
arXiv AI
Sep 11

Belief-State Engine: Augmenting LLMs for Principled Planning Under Partial Observability

The paper introduces the Belief-State Engine (BSE), an inference module that supplies a large language model (LLM) with a Bayesian posterior over hidden states in a partially observable Markov decision process (POMDP). By keeping the raw action‑observation log hidden from the LLM, the BSE ensures the agent behaves as a sound Markov policy on the belief MDP, thereby inheriting classical POMDP optimality guarantees. Experiments on the Tiger POMDP and a red‑team attack‑graph task show that BSE‑augmented agents outperform six baselines in task return, belief calibration, and decision consistency.

By Arnab Chattopadhayay, Debdipta Halder
arXiv AI
Sep 11

Self-Evolving Scientific Agent Designs Physically Reasoned White-Box Fluid Control

The paper presents a method where self‑evolving scientific agents design explicit, neural‑network‑free white‑box controllers for fluid dynamics tasks. By iteratively interpreting simulation data, the agents accumulate control knowledge and refine controller code, ultimately achieving robust control of an underactuated two‑joint swimmer in unsteady flows. The resulting controllers generalize across varying target positions, wake geometries, cylinder counts, and inflow speeds, and 2D control priors successfully transfer to accelerate 3D adaptation.

By Boai Sun, Wenjin Guo, Zongmin Yu, Liu Yang
arXiv Machine Learning
Sep 11

A Bellman Optimality Equation for Plasticity

The paper introduces a Bellman optimality equation for optimizing plasticity in Markov decision processes, extending recent work that reframes the stability‑plasticity dilemma as an empowerment‑plasticity tradeoff. It builds on Abel et al. (2025), which defined plasticity as the generalized directed information from observations to actions and empowerment as the reverse. This work is the first to address plasticity optimization under that new definition, providing a theoretical foundation similar to existing empowerment‑based approaches.

By Jeremy Lucas, Doina Precup
arXiv AI
Sep 11

Distributed Physical Layer Authentication and Collaborative RSMA in Non-Terrestrial Networks via Graph Reinforcement Learning

The paper introduces SAFA-MZ, a secure adaptive federated authentication scheme for multi‑zone non‑terrestrial networks that embeds group‑level authentication tags into a collaborative rate‑splitting multiple access transmission. It jointly optimizes high‑altitude platform station placement, user association, and power allocation to maximize secrecy spectral efficiency while meeting authentication reliability, power, and coverage constraints. The solution employs a repair‑based cross‑entropy method and a graph‑aware advantage actor‑critic algorithm, achieving up to 135% higher average secrecy spectral efficiency compared to single‑connect transmission.

By Parsa Rajabi, Mohammad Mirzaee, Mohammad Reza Abedi, Nader Mokari, Paeiz Azmi
arXiv Machine Learning
Sep 11

Topological Necessities: Mechanism-Invariant Strategic Subgoals for Cross-Embodiment Goal-Conditioned Control

The paper introduces topological necessities—mechanism‑invariant subgoals derived from the topology of successful trajectories—used to guide long‑horizon goal‑conditioned reinforcement learning. By computing homology in dimensions 0 and 1 over a transport‑weighted carrier, the authors obtain an enumerable gate set that forms a recursive topological gate hierarchy. These certified gates transfer across different embodiments (e.g., from PointMaze to Ant and Humanoid) without retraining, achieving state‑of‑the‑art performance on several benchmark tasks.

By Hao Shi, Xi Li
arXiv AI
Sep 11

Decision Transformer for UAV-Mounted RIS-Assisted Dynamic D2D Communications

The paper investigates UAV‑mounted reconfigurable intelligent surface (RIS) assisted device‑to‑device (D2D) communication with stochastic link activation. It models UAV motion, attitude, time‑varying Rician angles, and angle‑dependent RIS reflection, and formulates a joint optimization of UAV trajectory, attitude, and RIS phases to maximize average sum rate under mobility, energy, and hardware constraints. The authors employ deep reinforcement learning and a Decision Transformer trained on expert trajectories from multiple scenarios, showing that zero‑shot transfer outperforms direct DRL transfer and that online fine‑tuning achieves competitive performance with fewer interactions.

By Yaxuan Liu
arXiv Machine Learning
Sep 11

Domain-Specific Hallucination Detection in Large Language Models

The paper introduces a multi‑signal pipeline for detecting hallucinations in large language models, combining fine‑tuned DeBERTa‑v3 classification, Monte Carlo Dropout uncertainty, and temperature‑scaled calibration. On the HaluEval benchmark it achieves high performance (F1 = 0.915, AUROC = 0.977) across QA, summarization, and dialogue, and shows that 25 % of training data yields 77 % of full‑data performance. The authors also demonstrate that applying Direct Preference Optimization to a Qwen2.5‑0.5B generator cuts hallucination rates from 85.5 % to 37.7 %, and that domain‑specific fine‑tuning (PubMedBERT on SciFact) outperforms general‑domain models for biomedical text.

By Varun Teja Chundru, Debasmita Biswas
Hugging Face Trending Papers
Sep 10

Domain-Specific Hallucination Detection in Large Language Models

The paper introduces a multi‑signal pipeline for detecting hallucinations in large language model outputs, combining fine‑tuned DeBERTa‑v3 classification, Monte Carlo Dropout uncertainty, and temperature‑scaled calibration. On the HaluEval benchmark it achieves strong performance (F1 = 0.915, AUROC = 0.977) and further improves accuracy to 93.2% with MC Dropout. The authors also demonstrate that applying Direct Preference Optimization to a Qwen2.5‑0.5B generator reduces hallucination rates from 85.5% to 37.7%, and show that domain‑specific fine‑tuning (PubMedBERT on SciFact) yields better results than general‑domain training.

Hugging Face Trending Papers
Sep 10

LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation

LOCUS is a post‑training technique that selects a task‑aware low‑rank adaptation subspace to reduce token‑cost while preserving utility. By updating only a tiny fraction of parameters (0.24–0.28 %) on 3 B‑parameter backbones, LOCUS cuts continuation length by up to 39.84 % on Pythia‑2.8B and 14.87–17.58 % on Qwen2.5‑3B, without materially affecting the internal preference diagnostic.

Hugging Face Trending Papers
Sep 10

Negative Self-Distillation: Learning to Reason by Avoiding Flaws

Negative Self-Distillation (NSD) is a new framework for improving large language models by encouraging them to diverge from their own flawed reasoning rather than imitate privileged solutions. Unlike On-Policy Self-Distillation (OPSD), which can suppress uncertainty and exploratory behavior, NSD generates a question‑specific negative condition (e.g., a careless reasoner) and pushes the student’s distribution away from it. A dynamic gating mechanism isolates reasoning‑critical tokens so that only behavioral flaws are penalized, preserving linguistic capabilities, and empirical results show NSD consistently outperforms OPSD and other label‑free self‑bootstrapping RL baselines.

Hugging Face Trending Papers
Sep 10

Harnessing Intrinsic Subject-Aware Attention for Controllable Multi-Subject Video Generation

The paper tackles two main issues in multi-subject video generation—uncontrollable fidelity strength and semantic drift—by studying Diffusion Transformers (DiTs). It discovers that certain attention blocks naturally create an Intrinsic Spatial Grounding Map (ISGM) that accurately locates reference subjects. Leveraging this insight, the authors introduce Dual-phase Intrinsic Attention Leveraging (DIAL), which uses ISGM during low-noise stages to control fidelity strength without retraining and during high-noise stages to generate preference pairs for reinforcement learning, thereby anchoring attention and reducing semantic drift. Experiments on the OpenS2V-Eval benchmark show that DIAL outperforms baseline models, improving identity consistency and enabling controllable fidelity strength.

Hugging Face Trending Papers
Sep 10

Generative Replay Mitigates Sample Starvation in Quantum Architecture Search

The paper introduces GenQAS, a tensor‑network‑guided reinforcement learning framework that uses a learned local transition model to generate synthetic circuit transitions for prioritized generative replay. By mixing these synthetic transitions with real experience during Double Deep Q‑Network updates, GenQAS addresses sample starvation in quantum architecture search. Experiments on chemical Hamiltonian benchmarks up to 12 qubits and a 15‑qubit Ising model show significant improvements in success probability and circuit compactness, while a noisy 6‑qubit BeH₂ transfer experiment demonstrates a 92.7% reduction in steps to chemical accuracy.

Hugging Face Trending Papers
Sep 10

Fork Where the Model Changes Its Mind: Belief-Shift Branching for Tree-Structured Reinforcement Learning

The paper introduces belief‑shift branching, a method for placing forks in tree‑structured reinforcement learning rollouts by identifying points where a model’s answer belief changes most. Unlike traditional structural or entropy‑based approaches, belief‑shift uses a probe, logit‑lens depth profile, or learned activation direction to locate pivots in the value curve, incurring minimal computational overhead. Experiments across multiple models and benchmarks show that belief‑shift forking consistently outperforms baseline methods, yielding significant gains in mathematics and code tasks.

arXiv Machine Learning
Sep 10

Miles v0.1: Production-Level Post-Training

arXiv:2609.08368v1 Announce Type: new Abstract: We present Miles v0.1, a full-stack, production-ready system for frontier post-training. Building upon the clean design of slime, Miles designs each st...

By RadixArk, :, Tom Chen, Mao Cheng, Shi Dong, Kangrui Du, Yanbin Jiang, Jiajun Li, Yiming Li, Tao Lin, Yusheng Su, Andy Ye, Yueming Yuan, Zhichen Zeng
arXiv AI
Sep 10

Flow3D-OPD: Multi-Teacher On-Policy Distillation for 3D Geometry Generation with Flow-Matching Diffusion Transformer

arXiv:2609.07137v1 Announce Type: cross Abstract: Recent image-to-3D generation models built on flow-matching diffusion Transformers (DiT) can produce high-fidelity meshes, yet their post-training st...

By Zhiwei Ning, Zhen Zhou, Puhua Jiang, Xintong Han, Gengming Zhang, Jie Yang, Zhonglong Zheng, Yuanjie Zheng, Wei Liu, Chunchao Guo