Microsoft
The technology underlying ChatGPT has the potential to do a lot more than just talk. Linxi “Jim” Fan, an AI researcher at chipmaker Nvidia, has been working with some colleagues to find a way to get rid of the powerful GPT-4 language model – the “brains” behind ChatGPT and a growing number of other apps and services blocky video game minecraft.
The Nvidia team, which included Anima Anandkumar, the company’s director of machine learning and a professor at Caltech, has developed a Minecraft bot called Voyager that uses GPT-4 to solve in-game problems. The language model generates objectives that help the agent explore the game and code that improves the bot’s in-game capabilities over time.
Voyager doesn’t play the game like a human, but can read the game’s status directly via an API. For example, it might see a fishing rod in its inventory and a river nearby, and use GPT-4 to suggest a target to do some fishing to gain experience. This goal is then used for GPT-4 to generate the code necessary for the character to achieve that goal.
The most novel part of the project is the code that GPT-4 will generate to add behaviors to Voyager. If the originally proposed code doesn’t work perfectly, Voyager will attempt to refine it using error messages, feedback from the game, and a description of the code generated by GPT-4.
Over time, Voyager builds a code library to learn to create increasingly complex things and explore more of the game. A chart created by the researchers shows how powerful it is compared to other Minecraft agents. Voyager acquires more than three times the items, explores more than twice as far, and builds tools 15 times faster than other AI agents. According to Fan, the approach could be improved in the future by adding a way to the system to integrate visual information from the game.
Advertising
While chatbots like ChatGPT have wowed the world with their eloquence and seeming knowledge – albeit often making things up – Voyager shows the enormous potential of language models to perform helpful actions on computers. Using language models in this way could potentially automate many routine office tasks, potentially representing one of the greatest economic impacts of the technology.
The process Voyager uses with GPT-4 to figure out how to do things in Minecraft could be adapted for a software wizard figuring out how to automate tasks via the operating system on a PC or phone. OpenAI, the startup that developed ChatGPT, added “plugins” to the bot that allow it to interact with online services like grocery delivery app Instacart. Microsoft, the owner of Minecraft, also trains AI programs to play it, and the company recently announced Windows 11 Copilot, an operating system feature that uses machine learning and APIs to automate certain tasks. It can be a good idea to experiment with this type of technology in a game like Minecraft, where buggy code can do relatively little harm.
Of course, video games have long been a testing ground for AI algorithms. AlphaGo, the machine learning program that dominated the extremely subtle board game Go in 2016, made its name by playing simple Atari video games. AlphaGo used a technique called “reinforcement learning” that trains an algorithm to play a game by giving it positive and negative feedback, such as from within-game scores.
Using this method, it is more difficult to guide an agent in an open-ended game like Minecraft, where there is no score or set objectives, and where a player’s actions may not pay off until much later. Regardless of whether you think we should prepare now to contain the existential threat posed by AI, Minecraft appears to be an excellent playground for this technology.
This story originally appeared on wired.com.
