arXiv Machine Learning By Genki Kinoshita, Ko Nishino

BAL: Bidirectional Autoregression in Latent Space for Learning Human Movement Representations

Read the original on arXiv Machine Learning →

BAL (Bidirectional Autoregression in Latent Space) is a self‑supervised framework that learns human movement representations by predicting future and past latent states, mirroring how humans anticipate actions. It decomposes predictions into global sequence‑level semantics and local context‑independent movement, ensuring non‑trivial training signals and preventing collapse. Experiments on large datasets show BAL outperforms baselines in motion captioning, forecasting, and interpolation tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI
3d ago

SALA: Semantically Aligned Latent Actions Enable Dual-Axis Scaling of Embodied Foundation Models

SALA introduces a semantically aligned latent action framework that bridges human and robot video data without requiring paired examples. By leveraging language descriptions to estimate semantic similarities, SALA guides human latent actions toward related robot latents while grounding the latent space with robot action reconstruction. This shared latent action space enables dual‑axis scaling: larger general human pre‑training reduces robot action prediction error, and task‑specific human post‑training boosts real‑world robot success.

By Jiachun Jin, Zetong Zhou, Haonan Wang, Xiao Yang, Jun Zhu, Zhijie Deng
arXiv Computer Vision
Sep 28

WALT: Learning World-Model-Aligned Latent Trajectories for Autonomous Driving

WALT introduces a method to align latent trajectories with pretrained driving world models, creating a compact generative trajectory space that preserves action-relevant semantics without altering the original model. The approach uses a dual-branch autoencoder to map raw waypoints into this latent space and transfers visual world knowledge into trajectory representations. Experiments on NAVSIM benchmarks show modest performance gains and a 30.5% reduction in planner FLOPs, indicating that maintaining world representations while extracting action-relevant information can improve trajectory planning efficiency.

By Mingkai Jia, Jiaxin Guo, Zhijian Shu, Jiawei Xu, Mingxiao Li, Jintao Cheng, Ping Tan, Wei Yin
arXiv Computer Vision
Oct 23

Learning from Next-Frame Prediction: Towards Unified and Scalable Visual Representation Learning

The paper introduces VANE, a self‑supervised visual pretraining framework that learns from next‑frame prediction in embedding space. It uses a causal encoder to summarize past observations, a state predictor to infer the representation of the entire next frame, and a lightweight generative decoder to reconstruct fine‑grained spatial detail. By applying the same objective to both videos and still images through a local‑to‑global crop sequence, VANE achieves state‑of‑the‑art performance on anticipation, recognition, and dense perception benchmarks, notably outperforming V‑JEPA on EPIC‑KITCHENS‑100 action anticipation.

By Yifeng Yao, Jinghan Li, Zhihao Zhang, Yajiao Xiong, Yang Jin, Kun Xu, Huishuai Zhang, Dongyan Zhao