Manipulation, locomotion, sim-to-real transfer and autonomous driving: learning systems that have to survive physics.
arXiv:2608.28518v1 Announce Type: cross Abstract: We investigate whether automatic speech recognition (ASR) errors in user input can lead to unsafe outputs from Embodied AI (EAI) models. We find that...
CEDAR is a counterexample-guided framework that translates natural-language instructions for embodied agents into regular languages over environment event traces, represented as deterministic finite automata. By using a language model for semantic judgments and execution traces for correction, CEDAR turns constraints into executable finite-state objects, enabling the intersection of learned skills with additional specifications. In Minecraft experiments, CEDAR preserves temporal and spatial constraints better than a program-generating baseline and reduces cumulative LLM queries by reusing learned skills.
The paper investigates how scaling video diffusion models affects performance on autonomous driving data, training models ranging from 1 M to 9 B parameters on up to 5,500 hours of driving footage. Validation loss follows predictable power‑law trends with both model size and training exposure, showing that longer training yields faster loss reduction than increasing model size, though larger models still achieve lower asymptotic loss. Guided by these findings, the authors train a 9 B‑parameter model that becomes the largest video diffusion model trained from scratch on driving data and sets a new open‑source state‑of‑the‑art on the nuScenes benchmark.
uScenes is a new multimodal dataset for underwater robot perception that provides synchronized 3D multibeam sonar point clouds and RGB imagery. It comprises 110 scenes with 95,834 observations, totaling 277.6 minutes of data collected during multiple field sessions. The dataset aims to support research in underwater sensor fusion, cross‑modal representation learning, and 3D scene understanding.
arXiv:2608.27997v1 Announce Type: new Abstract: Air-ground cross-view referring person detection is a necessary component in the language-to-perception-to-control chain of collective embodied intelli...
The paper introduces OccLinker, a lightweight plugin for vision‑based occupancy networks that reduces flickering by efficiently merging historical static and motion cues with current features via a dual cross‑attention mechanism. It generates correction components to refine base network predictions and proposes a new temporal consistency metric to quantify flickering. Experiments on two benchmark datasets show that OccLinker improves performance with minimal computational overhead while effectively diminishing flickering artifacts.
The paper introduces Synthetic Linguistic Agency (SLA), a framework that defines linguistic agency in terms of embodiment, participation, and precariousness. It presents two studies: one that operationalizes SLA criteria and identifies existing systems, and another that builds an Embodied Mortal Agent (EMA) using mortality‑grounded reinforcement learning. Experiments show the EMA’s linguistic choices depend on its body and social history, influence partner behavior, and adapt over time, demonstrating SLA in an artificial agent.
Iron is a new framework for training generalist virtual agents that aligns low‑level actions with high‑level intents using a stepwise cycle‑consistent reward. It also repurposes failed trajectories through a hindsight reproduction mechanism to improve learning efficiency and task diversity. Experiments show Iron‑trained agents outperform those trained with three times more data, achieving a 25.06% relative improvement on unseen web tasks and better performance on complex tasks.
OceanGym is the first comprehensive benchmark for underwater embodied agents, featuring eight realistic task domains and a unified agent framework powered by Multi‑modal Large Language Models (MLLMs). It challenges agents to process optical and sonar data, navigate complex environments, and achieve long‑horizon goals amid low visibility and dynamic currents. Experiments show significant performance gaps between current MLLM agents and human experts, underscoring the difficulty of perception, planning, and adaptability in ocean settings.
arXiv:2608.08273v2 Announce Type: replace-cross Abstract: Vision-based embodied agents executing multi-step natural language instructions require feedback mechanisms that assess task progress over co...
The paper proposes a label‑free 3D perception framework where roadside units (RSUs) act as unsupervised teachers for self‑driving cars. RSUs learn local 3D detectors from unlabeled data and broadcast predictions to passing vehicles, which use these as pseudo‑labels to train an ego‑centric detector. In a CARLA simulation, the method achieves 82.3% AP for vehicle detection, approaching a fully supervised upper bound of 94.4%, and demonstrates scalability and complementarity with existing ego‑centric approaches.
The paper exposes a new vulnerability in Federated Retrieval-Augmented Generation (FedRAG) called Routing Hijacking, where a malicious client forges its semantic profile to attract target queries despite lacking relevant data. Experiments across three FedRAG routing architectures show that this attack consistently misroutes queries, causing downstream failures such as missing evidence, poisoning, incorrect answers, hallucinations, and sycophantic behavior. The authors propose a trust‑aware post‑routing framework that reweights clients based on evidence feedback, which effectively suppresses persistent hijacking and transfers to a learned neural router.
arXiv:2607.22494v2 Announce Type: replace-cross Abstract: Collision anticipation in autonomous driving requires not only accurate early warnings but also interpretable reasoning about what risk facto...
arXiv:2608.28386v1 Announce Type: new Abstract: Existing monocular full-body 3D human-object interaction (HOI) methods do not combine explicit finger-level grasp optimization with category-agnostic o...
arXiv:2506.07188v2 Announce Type: replace Abstract: End-to-end neural networks have become a dominant paradigm in autonomous driving, where reliable deployment requires controllable post-training ada...
CNeo-Bench is a new benchmark comprising 4,759 Chinese neologisms, each with reference definitions and categorized by linguistic mechanisms such as phonetic substitution and visual character decomposition. The benchmark includes a two-tier evaluation framework that tests whether models can describe a neologism and whether they can manipulate its underlying mechanism. Evaluation of 18 large language models shows that most perform poorly on definition generation (below 40%) and exhibit a recognition‑manipulation gap, often paraphrasing rather than restoring the original form; few‑shot prompting helps but does not fully resolve the errors.
The paper introduces the KITScenes LongTail dataset, a curated collection of rare driving scenarios designed to evaluate how well reasoning models in autonomous driving follow their own reasoning. The authors find that many current models frequently diverge between the actions they state in their reasoning chains and the actions they actually execute, a phenomenon they term incoherence. They show that when the reasoning and execution disagree, the reasoning is often correct, and enforcing coherence via a kinematic model can improve motion planning, indicating that coherent action based on stated reasoning is essential for trustworthy autonomous driving.
SUFLECA is a weakly supervised framework that improves zero‑shot CAD‑to‑image alignment by scaling geometry‑grounded feature learning using Normalized Object Coordinates across up to 12 real and synthetic datasets. It introduces a geometrically consistent matching algorithm that reliably establishes CAD‑to‑image correspondences, enabling accurate, sub‑second alignment without iterative pose refinement. On the ScanNet25k benchmark, SUFLECA achieves 32.8%/42.6% category/instance accuracy, outperforming the strongest zero‑shot baseline by 9.7/12.5 percentage points and surpassing existing pose‑supervised methods for the first time.
The paper introduces TaintedPixels, a proactive video‑protection technique that embeds subtle, structured perturbations into the blue channel of facial regions. These perturbations remain invisible in the original video but become noticeable after black‑box manipulation, thereby flagging deepfakes. Experiments on three off‑the‑shelf manipulation tools and two detectors show that TaintedPixels achieves the highest forgery detection rate while keeping visual distortion minimal (LPIPS = 0.0042), and a human study confirms that protected videos are rarely suspected, whereas forgeries from protected sources are identified as fake far more often than those from unprotected sources.
The paper introduces the Persuasion Index (PI), a taxonomy of 15 persuasion dimensions grounded in psychological and communication theories, implemented with 55 lexicon- and rule-based sub-features. PI is modular, allowing individual detectors to be swapped while preserving its theoretical framework. Evaluations on four English argumentative datasets show that PI provides a shared, lightweight feature space that captures meaningful predictive signals and reveals consistent dimension-level associations with persuasion outcomes, with variations across topics and stances.