← Back to all news
arXiv AI By Tan Su, Haoxiang Yang, Ruxin Wang, Binghui Xie

MemCorr-DP: Counterfactual Correspondence Conditioning for a Diffusion Policy Guided by a Reference

Read the original on arXiv AI →

MemCorr-DP is a diffusion policy that uses counterfactual correspondence conditioning to align a frozen RoMa v2 reference trajectory with the current scene in explicit 3D relations. By assigning opposite behaviors the same physical state and noisy action while retaining reference-specific denoising targets, and then fine‑tuning with mixed‑condition data, the policy learns to adapt from ground‑truth geometry to measured correspondence errors. In a door‑opening task with large spatial and viewpoint shifts, MemCorr‑DP achieves 96.67% closed‑loop success versus 88.00% for a visual Transformer, demonstrating that explicit 3D reference relations provide robust conditioning when spatial and viewpoint changes are compounded.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Oct 19

When Plans Change, Remember: Replanning-Guided Visual Memory for Vision-Language-Action Models

The paper introduces RePlanMem-VLA, a selective visual memory system for Vision‑Language‑Action models that uses a policy’s own plan revisions as an online memory‑write signal. By detecting unusually large disagreements between temporally aligned action predictions, the method identifies task‑relevant events and updates a bounded visual memory while suppressing redundant frames. Experiments on MIKASA‑Robo‑VLA tasks and real‑robot replay data show that RePlanMem‑VLA retains essential observations in every episode and achieves high ROC‑AUC scores for event detection.

By Victoria Smelova, Daniil Kazachkov, Georgiy Kazantsev, Iuliia Tsvetkova, Egor Cherepanov, Nikita Kachaev, Artem Latyshev, Aleksandr I. Panov, Alexey K. Kovalev
llmsroboticsmultimodal
More like this →
arXiv Computer Vision
Oct 12

ForeAct3D: Policy-Grounded Future World Modeling for VLA Policies

ForeAct3D is a framework that grounds future world modeling in Vision‑Language‑Action policies by decoding depth, semantic segmentation, and camera pose from policy representations into current and future 3D scene states. The future queries are conditioned on the policy‑generated action chunk, and a physical‑consistency closure enforces background staticity, instance‑level rigidity, and wrist‑camera pose alignment with end‑effector kinematics. This approach eliminates the need for future prediction at inference and achieves high success rates on LIBERO and CALVIN, outperforming the base policy across all suites.

By Zhe Tao, Feiran Wang, Gaowen Liu, Ramana Rao Kompella, Yan Yan
computer-visionroboticsmultimodal
More like this →
arXiv AI
Sep 21

Outcome-Conditioned End-Effector Geometry Across Vision-Language-Action Policies

The study examines how different vision‑language‑action (VLA) policies execute a manipulation task by comparing the geometry of their end‑effectors across 15,000 closed‑loop LIBERO rollouts. By pairing 3,600 configuration‑matched policy executions, the authors find that when both policies succeed, their end‑effector trajectories are much closer (median DTW distance 0.0120 m) than when only one succeeds (0.0380 m), a pattern consistent across all tasks, policy pairs, and nine representations. Even successful executions remain as far from same‑task demonstrations as the demonstrations are from each other, indicating that task‑associated geometry, rather than training data overlap, drives these differences.

By Xingyu Lin, Zhuang Li, Zhongrun Wu, Shouquan Zhou, Dehui Du
roboticsmultimodal
More like this →