AgentDreamer is an agent-guided optimization framework that aligns text instructions with simulated 3D motion by using an off-the-shelf agent to generate a structured semantic motion prior. The prior specifies what should move, in which direction, and in what temporal order, and is represented as parametric functions over temporal chunks to produce dense 3D trajectories. These trajectories are then used to optimize a differentiable MPM simulator so that its rollout matches the prior while respecting scene dynamics, and the framework introduces Motion-IG, an information‑theoretic metric for measuring text‑motion alignment.
arXiv:2607. 01766v1 Announce Type: new Abstract: LLM agents are increasingly used to translate natural language into 3D scenes in a procedural way, but existing systems focus on static output.
LLM agents are increasingly used to translate natural language into 3D scenes in a procedural way, but existing systems focus on static output. Dynamic 4D scenes from text alone, in which liquids flow, particles emit, rigid bodies cascade, and articulated mechanisms move, remain largely unexplored despite their value as editable content and as physics-grounded training data for video generation and embodied AI.
InteractAgent is a multimodal large language model system designed for long‑horizon human‑scene interaction. It plans, executes, observes, and reflects in a closed loop, using a single MLLM for planning and evaluation and a frozen MotionLLM for execution. The system incorporates online observation for step‑wise recovery decisions and memory‑augmented reflection for multi‑round plan refinement, achieving improved task success and reduced penetration on the UniHSI benchmark compared to training‑free baselines.
Synthesizing human motion from textual descriptions is essential for immersive digital applications, yet existing methods face a persistent trade-off between semantic fidelity and physical realism. Large language model (LLM)-based approaches can interpret diverse open-vocabulary instructions and compose high-level action plans, but they often generate motions that violate physical constraints.
arXiv:2606. 26981v1 Announce Type: cross Abstract: Synthesizing human motion from textual descriptions is essential for immersive digital applications, yet existing methods face a persistent trade-off between semantic fidelity and physical realism.
arXiv:2605. 14398v2 Announce Type: replace Abstract: World models have emerged as a powerful paradigm for building interactive simulation environments, with recent video-based approaches demonstrating impressive progress in generating visually plausible dynamics.
The paper introduces Latent Motion Reasoning (LMR), a two‑stage approach that separates text‑to‑motion generation into a planning phase and an execution phase. LMR uses a Dual‑Granularity Tokenizer to create a compressed, semantically rich reasoning latent for global trajectory planning and a high‑frequency execution latent for detailed motion fidelity. Experiments on T2M‑GPT and MotionStreamer show that this architecture improves both semantic alignment and physical plausibility compared to direct translation methods.
arXiv:2608.23258v1 Announce Type: new Abstract: We propose HetSkills, a novel framework designed to progressively learn heterogeneous skills within a unified latent space for physics-based character...
FilmFlow is an agentic framework that transforms natural‑language instructions or reference videos into explicit camera trajectories for dynamic 3D environments. It uses a Director agent that composes high‑level motions from concurrent atomic operations, supported by Interpreter, Scout, and Critic agents for intent extraction, viewpoint selection, and iterative refinement. The authors also introduce CinemaBench, a benchmark of 150 Blender scenes with 750 text‑directed tasks and 60 video‑reference tasks, where FilmFlow outperforms baselines and receives the highest user preference.
We propose HetSkills, a novel framework designed to progressively learn heterogeneous skills within a unified latent space for physics-based character control. The core idea is to treat this latent sp...
arXiv:2610.20560v1 Announce Type: cross Abstract: 3D object composition is pivotal to digital content creation and robot simulation, while also serving as a critical testbed for physical intelligence...
arXiv:2608.20699v1 Announce Type: new Abstract: Animating articulated 3D meshes via text requires satisfying strict kinematic constraints, modeling causal interactions between parts, and achieving in...