what you need to know
- Google has unveiled its latest artificial intelligence, SIMA, a “generalist agent” designed to help users complete tasks in video games.
- SIMA is language-based, meaning it requires what the user sees on the screen and their instructions to complete tasks such as gathering resources.
- DeepMind says it has used nine games to train SIMA so far, but more work remains before it can handle complex tasks and instructions.
Google DeepMind has announced its latest project, a “generalist artificial intelligence agent” designed to help users perform tasks while playing games.
According to DeepMind, the latest artificial intelligence is called “Scalable Instructable Multiworld Agent”, or “SIMA” for short. Google’s division focused on artificial intelligence says SIMA can “sense and understand a variety of environments and then take action to achieve directed goals.”
DeepMind added that in order to work, SIMA needs what the user sees as well as “natural language” instructions provided by the user. The AI is said to use the user’s standard keyboard and mouse input to move the user’s character around the game world. By extension, the post adds that SIMA can “interact with any virtual environment.”
SIMA has tested 600 basic skills, such as turning left, climbing a ladder, opening the game pause menu for settings, and more. Google says SIMA can also perform “simple tasks” in 10 seconds. Some of these tasks involve driving a car in Goat Simulator 3 and telling SIMA to walk to your spaceship in No Man’s Sky.
To get SIMA to its current level, Google said it partnered with game developers such as No Man’s Sky maker Hello Games and Teardown maker Tuxedo Labs. SIMA also teaches using an environment called the Build Lab built in the Unity engine. Through this process, SIMA learned how to manipulate objects and better understand the physical world in video games.
Google also relied on real-life gamers as an initial way to understand how SIMA works. They monitored them as a pair, with one player playing the game while the other gave them instructions on how to complete the task.
Google’s DeepMind trained SIMA using a total of nine different games, all of their own genre. During the course of the research, the department found that AI agents trained on multiple games delivered better results than systems trained on one game. More research shows that SIMA’s reliance on language is critical to its performance.
Without user input, SIMA is said to operate “in an appropriate but aimless manner.” DeepMind observed SIMA exploring and gathering resources, but not following the game’s objectives unless strictly told to do so.
SIMA is still in its infancy, but Google seems confident in its “language-driven” capabilities to help gamers. What’s more, more research is reportedly necessary because Google wants SIMA to be able to understand “high-level language instructions to achieve more complex goals.”