Preprint
ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation
ArXiv.org
07/09/2020
DOI: 10.48550/arXiv.2007.04954
Abstract
We introduce ThreeDWorld (TDW), a platform for interactive multi-modal physical simulation. With TDW, users can simulate high-fidelity sensory data and physical interactions between mobile agents and objects in a wide variety of rich 3D environments. TDW has several unique properties: 1) realtime near photo-realistic image rendering quality; 2) a library of objects and environments with materials for high-quality rendering, and routines enabling user customization of the asset library; 3) generative procedures for efficiently building classes of new environments 4) high-fidelity audio rendering; 5) believable and realistic physical interactions for a wide variety of material types, including cloths, liquid, and deformable objects; 6) a range of "avatar" types that serve as embodiments of AI agents, with the option for user avatar customization; and 7) support for human interactions with VR devices. TDW also provides a rich API enabling multiple agents to interact within a simulation and return a range of sensor and physics data representing the state of the world. We present initial experiments enabled by the platform around emerging research directions in computer vision, machine learning, and cognitive science, including multi-modal physical scene understanding, multi-agent interactions, models that "learn like a child", and attention studies in humans and neural networks. The simulation platform will be made publicly available.
Details
- Title: Subtitle
- ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation
- Creators
- Chuang GanJeremy SchwartzSeth AlterMartin SchrimpfJames Traer - University of Iowa, Iowa Neuroscience InstituteJulian De FreitasJonas KubiliusAbhishek BhandwaldarNick HaberMegumi SanoKuno KimElias WangDamian MrowcaMichael LingelbachAidan CurtisKevin FeigelisDaniel M BearDan GutfreundDavid Cox - Stanford University School of MedicineJames J DiCarloJosh McDermottJoshua B TenenbaumDaniel L. K Yamins
- Resource Type
- Preprint
- Publication Details
- ArXiv.org
- DOI
- 10.48550/arXiv.2007.04954
- ISSN
- 2331-8422
- Language
- English
- Date posted
- 07/09/2020
- Academic Unit
- Iowa Neuroscience Institute; Psychological and Brain Sciences
- Record Identifier
- 9984071668502771
Metrics
15 Record Views