arXiv Machine Learning

Runtime-Incremental Transformer for Reinforcement-Learning-Based Adaptive Control

arXiv Machine Learning
Sep 16

TARC: Time-Adaptive Robotic Control

TARC (Time‑Adaptive Robotic Control) is a reinforcement‑learning framework that lets a policy predict both a control action and how long it should be applied, thereby learning temporally extended actions. By optimizing task performance under constraints on the number of control switches, TARC can adapt its control rate online, using high‑frequency feedback only when necessary. Experiments on a high‑speed RC car, a Unitree Go1 quadruped, and a vision‑language action model show that TARC matches the performance of high‑frequency discrete‑time controllers while operating at less than half their control frequency.

By Arnav Sukhija, Lenart Treven, Jin Cheng, Florian D\"orfler, Stelian Coros, Andreas Krause
arXiv Machine Learning
Jun 25

Memory-Efficient Policy Libraries with Low-Rank Adaptation in Reinforcement Learning

arXiv:2606. 25700v1 Announce Type: new Abstract: When fine-tuning Large Language Models (LLMs), there has been success in minimizing both memory usage and computation with Parameter-Efficient Fine-Tuning (PEFT), like Low Rank Adaptation (LoRA).

By Samuel Valland Lyngset, Tor Viljen Raanaas, Gard Sveipe, Eirik M{\o}ller Nilsen, Jim Torresen, Kai Olav Ellefsen, Tobias L{\o}mo
arXiv Machine Learning
Jul 13

Learning More from Less: Reinforcement Learning from Hindsight

arXiv:2607. 09042v1 Announce Type: new Abstract: Reinforcement learning (RL) is increasingly used to post-train vision-language-action (VLA) models, but every update consumes robot rollouts that are slow and costly to collect, making sample efficiency a central concern.

By Iris Xu, Sunshine Jiang, John Marangola, Nitish Dashora, Richard Li, Thomas Liu, Zexue He, Yuheng Zhi, Alex Pentland, Pulkit Agrawal, Zhang-Wei Hong
arXiv AI
Jul 24

VPWEM: Non-Markovian Visuomotor Policy with Working and Episodic Memory

arXiv:2603. 04910v2 Announce Type: replace-cross Abstract: Imitation learning from human demonstrations has achieved significant success in robotic control, yet most visuomotor policies still condition on single-step observations or short-context histories, making them struggle with non-Markovian tasks that require long-term memory.

By Yuheng Lei, Zhixuan Liang, Hongyuan Zhang, Ping Luo
arXiv AI
6d ago

Divide-and-Remember: Recursive Action-Relevant Memory for Long-Horizon VLA Policies

The paper introduces Divide-and-Remember (D&R), a recursive memory method for vision-language-action (VLA) policies that optimises memory by maximizing the conditional mutual information between actions and memory given observations. D&R recursively divides the full history into top‑K selections over 2K tokens, using a shared lightweight selector across all recursion blocks to handle unbounded histories efficiently. Evaluated on the RoboMME benchmark of 16 long‑horizon manipulation tasks, D&R achieves state‑of‑the‑art success rates with consistent gains across all suites while using only 64 tokens, and similar improvements are observed in real‑robot experiments.

By Xuehui Yu, Eason Yu, Meiyi Wang, Haozhe Du, Stefano V. Albrecht, Harold Soh
arXiv AI
Sep 25

Policy Complexity, Reaction Time, and Bounded Rationality in Reinforcement Learning

The paper introduces MI‑SARSA, an on‑policy temporal‑difference algorithm that incorporates mutual‑information regularization to model bounded rationality in reinforcement learning. By penalizing state‑specific deviations from a learned marginal action prior, the algorithm selectively uses state information only when the expected return outweighs the informational cost, yielding a reward‑complexity tradeoff. MI‑SARSA also predicts reaction times, showing that stronger information penalties lead to simpler policies, lower control costs, and faster responses, while regularization mitigates performance loss after environmental shifts at the expense of asymptotic return.

By James Wu, Chris R. Sims