MIT News AI

AI agents create virtual playgrounds to help robots get crucial training data

“SceneSmith” system uses collaborative AI agents to create realistic 3D environments of places like kitchens, hotels, and living rooms, where robots can simulate everyday chores.

Hugging Face Trending Papers
Aug 19

Beyond Placement and Articulation: Usage-Driven Code Scenes for Embodied Interaction

The paper introduces RoomWright, a code‑driven framework that generates 3D indoor scenes for embodied AI by focusing on functional usage rather than just visual layout. It performs usage‑driven object reasoning, treating anchors as task centers to select task‑required objects and their affordances, and compiles interactions into trigger‑condition‑effect rules that update object states. The system also addresses ambiguous object orientation through annotation‑guided usage cues, producing scenes that are executable, editable, and ready for simulation‑based policy learning.

arXiv AI
Aug 13

Towards the Harness of Embodied Agents

arXiv:2608. 11246v1 Announce Type: new Abstract: The success of coding agents has established the harness as a paradigm: what an agent achieves depends not on the model alone, but on the infrastructure around it.

By Qi Wang, Tianyi Wang, Chengyang Li, Shikun Ban, Yurun Chen, Yizhong Ge, Jason Qin, Chengtai Li, Wentao Zhu
Microsoft Research
Aug 3

Orchard: An open framework for scalable agentic AI

Orchard is an open-source framework for the research community to train and evaluate AI agents across task types. It reduces complexity while supporting strong performance from smaller models by enabling researchers to reuse the same infrastructure.

By Baolin Peng, Wenlin Yao, Qianhui Wu, Hao Cheng, Jianfeng Gao
arXiv AI
Aug 20

Emergence of Agentic AI: A Review on Evolution, Background, Working Principles, Applications, Adoption Factors, and Future Research Directions

The article reviews the emergence of Agentic AI, covering its evolution, theoretical foundations, working principles, and architectural aspects. It surveys recent scholarly contributions across various domains, highlighting real‑world applications, current research findings, and existing challenges. The review also proposes a framework for stakeholder adoption and outlines future research directions to guide researchers and practitioners.

By AKM Bahalul Haque, Al Amin Islam Ridoy, Mohammad Rayhan, Ivan Porres
arXiv AI
Sep 25

World Action Agent: Harnessing VLMs for Robot Manipulation via World Action Rehearsal

World Action Agent (WAA) is a multi‑agent framework that lets vision‑language models (VLMs) directly pilot robots by operating within a visual action workspace. The workspace provides automatically selected contact views, editable action rehearsals, and in‑view correction to refine decisions before low‑level execution. WAA learns procedural skills from expert videos and human teaching, and its interaction traces can train smaller VLMs, achieving state‑of‑the‑art success on LIBERO‑Pro and improving out‑of‑domain performance on robosuite and Qwen3.5‑9B.

By Yehang Zhang, Haojian Huang, Yifan Chang, Jianchong Su, Bohan Zhou, Yingjie Xu, Wosong Chen, Tianhao Zhou, Chenxu Wang, Tianyi Zhang, Yangkai Wei, Wenqian Li, Shiyuan Deng, Yinchuan Li, Ying-Cong Chen, Zexi Li
arXiv Computer Vision
Oct 2

LiteReality-Agent: An Agentic System for Interactable 3D Indoor Scene Reconstruction

LiteReality-Agent is an agentic system that reconstructs real indoor environments into realistic, articulated, and simulation-ready 3D scenes from RGB‑D scans. It treats reconstruction as a coding problem, where a coding agent iteratively edits a Python script (Room.py) using specialized tools, while an observe‑edit‑verify harness ensures evidence gathering, measurement, verification, layout optimisation, simulation readiness, and quality control. The system outperforms recent models like Astra and Fable in geometric accuracy, visual realism, and simulation compatibility, and its modular framework is positioned as a robust foundation for future agent‑driven reconstruction tasks.

By Zhening Huang, Yueyan Li, Johnathan Chiu, Xiaoyang Lyu, Matt Zhou, Yuxin Yao, Joan Lasenby, Shangzhe Wu