A paper describing MineDojo, Nvidia’s generalist AI agent that can accomplish steps from written prompts in Minecraft, won an Fantastic Datasets and Benchmarks Paper Award at the 2022 NeurIPS (Neural Information and facts Processing Devices) convention, Nvidia revealed on Monday.
To educate the MineDojo framework to participate in Minecraft, researchers fed it 730,000 Minecraft YouTube videos (with extra than 2.2 billion words and phrases transcribed), 7,000 scraped webpages from the Minecraft wiki, and 340,000 Reddit posts and 6.6 million Reddit reviews describing Minecraft gameplay.
From this facts, the scientists created a tailor made transformer design known as MineCLIP that associates video clip clips with specific in-recreation Minecraft actions. As a result, someone can inform a MineDojo agent what to do in the sport applying large-degree pure language, this sort of as “uncover a desert pyramid” or “build a nether portal and enter it,” and MineDojo will execute the series of steps necessary to make it come about in the recreation.
MineDojo aims to build a adaptable agent that can generalize uncovered steps and apply them to distinct behaviors in the match. As Nvidia writes, “When scientists have very long educated autonomous AI agents in movie-activity environments these kinds of as StarCraft, Dota, and Go, these agents are normally specialists in only a couple duties. So Nvidia scientists turned to Minecraft, the world’s most common match, to build a scalable instruction framework for a generalist agent—one that can productively execute a wide assortment of open up-ended responsibilities.”
The award-profitable paper, “MINEDOJO: Creating Open-Ended Embodied Brokers with World-wide-web-Scale Information,” debuted in June. Its authors involve Linxi Enthusiast of Nvidia and Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, and Anima Anandkumar of a variety of educational institutions.