The paper investigates when in‑context learning (ICL) can succeed even if the instruction or target task is not covered by a model’s pretraining data. It introduces a theoretical framework that distinguishes two types of distribution shift—unsupported instructions and unsupported target tasks—and shows how demonstrations can mitigate these shifts. The authors further extend the analysis to long‑horizon tasks that can be decomposed into pretraining‑supported subtasks, providing error bounds that explain how ICL can handle tasks beyond the model’s original support.
By Xuhan Tong, Yuchen Zeng, Jiawei Zhang
The paper introduces Minimally Invasive Steering Vector Optimization (MISVO), a method that adjusts a frozen language model’s final hidden states by adding vectors to steer outputs toward a test‑time reward while penalizing changes using the local KL geometry of the token distribution. MISVO derives an analytic Fisher term and a suffix score‑function term, showing that the suffix term is second‑order and that Fisher surrogates match the full KL gradient to first order. Experiments on preference and code‑generation tasks with 1B–14B parameter models demonstrate that MISVO achieves the highest mean reward in most settings while maintaining diversity and coherence comparable to Best‑of‑N.
By Taha Entesari, Jingyu Zhang, Daniel Khashabi, Mahyar Fazlyab
arXiv:2605.30117v2 Announce Type: replace
Abstract: Understanding how Vision-Language-Action (VLA) models transform multimodal knowledge into embodied control remains an open challenge. We present VL...
By Haoyuan Shi, Xiancong Ren, Yingji Zhang, Qinfan Zhang, Jiayu Hu, Haozhe Shan, Han Dong, Jinpeng Lu, Yinda Chen, Yi Zhang, Yong Dai, Xiaozhu Ju
arXiv:2507. 18043v2 Announce Type: replace-cross Abstract: Inference-time steering methods offer a lightweight alternative to fine-tuning large language models (LLMs) and vision-language models (VLMs) by modifying internal activations at test time without updating model weights.
By Duy Nguyen, Archiki Prasad, Elias Stengel-Eskin, Mohit Bansal
arXiv:2606. 12299v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models provide a natural language interface to robot control, but the mapping from language to behavior is often brittle and unintuitive: semantically similar instructions can induce drastically different behaviors, while some capabilities may not be elicitable through prompting alone.
By Hyun Joe Jeong, Gokul Swamy, Andrea Bajcsy
Iron is a new framework for training generalist virtual agents that aligns low‑level actions with high‑level intents using a stepwise cycle‑consistent reward. It also repurposes failed trajectories through a hindsight reproduction mechanism to improve learning efficiency and task diversity. Experiments show Iron‑trained agents outperform those trained with three times more data, achieving a 25.06% relative improvement on unseen web tasks and better performance on complex tasks.
By Jiahe Ying, Wendong Bu, Kaihang Pan, Bingchen Miao, Siyu Chen, Wen Wang, Xueming Jiang, Juncheng Li, Siliang Tang
arXiv:2602.01654v2 Announce Type: replace
Abstract: Steering vectors (SVs) offer a lightweight way to control large language models (LLMs) at inference time by shifting hidden activations, providing...
By Jiaqian Li, Yanshu Li, Kuan-Hao Huang
arXiv:2609.31665v2 Announce Type: replace
Abstract: Action anticipation predicts future human actions from partial video under incomplete context and temporal uncertainty. Recent systems introduce la...
By Mahsa Mohammadi, Zeyu Fu, Sareh Rowlands
arXiv:2606. 03093v1 Announce Type: new Abstract: Prompting steers large language models (LLMs) and vision-language models (VLMs) without weight updates, but it remains unclear how instruction changes reshape internal representations to produce behavior.
By Fan L. Cheng, Nikolaus Kriegeskorte
arXiv:2609.36691v1 Announce Type: new
Abstract: Manipulation behaviors vary widely across objects and scenes, but they share a small set of reusable skills, and planning with these skills helps embod...
By Jianshu Zhang, Ce Zhang, Xiyuan Yang, Chenwei Xu, Haoran Lu, Yijiang Li, Yaqi Xie, Katia P. Sycara, Han Liu
arXiv:2607. 06160v1 Announce Type: cross Abstract: Synthesizing long-context supervised fine-tuning (SFT) data is a scalable way to enhance the long-context understanding of large language models (LLMs), yet existing approaches share three limitations: narrow task coverage, insufficient instruction difficulty, and a lack of faithfulness supervision.
By Chenhao Yuan, Yinhao Xu, Shuwen Xu, Xizhi Yang, Jiaxiang Liu, Chenxi Zhou, Shaoping Huang, Haolin Ren, Pengfei Cao, Jun Zhao, Kang Liu
The paper introduces Intention Distillation (INDI), a method that injects behavior-level intent into Vision‑Language‑Action (VLA) model decoders by leveraging a frozen teacher vision‑language model to interpret demonstrations. During training, the teacher processes the current observation, instruction, coarse action summary, and execution video, producing a multimodal intent representation that the VLA decoder uses alongside trajectory and execution features to predict actions. Experiments on SimplerEnv‑Bridge, RoboCasa Kitchen, and real‑world tasks show that INDI consistently improves success rates, especially on longer‑horizon tasks, demonstrating that explicit modeling of semantic intent benefits action decoders.
By Sangoh Lee, Sangwoo Mo, Wook-Shin Han