BlizzCon 2016 DeepMind and StarCraft II Deep Learning Panel Transcript
StarCraft II and DeepMind
Kevin: So good morning, I’m Kevin Calderone. I am one of the engineers from the StarCraft II team that has been working with DeepMind on this project. Now StarCraft II is one of the most strategically deep games out there, and this makes it the perfect platform for developing AI that can tackle these types of problems. So together, up the last few months, we were working with DeepMind on building the tools needed to make this happen. And now that we have the core pieces of this built, we’d like to open up everything we’ve been working on.
Now with all this stuff, you should keep in mind that this is cutting-edge research. So it is hard for us to predict exactly where it will lead, but at some point we would love to apply the advancements made here back into our games, and make them better for our players. Now the type of AI we’re talking about here is very different than how it’s usually done in games.
Typically, all the AI behavior is custom-scripted by a game designer. This is the opposite of how deep learning works, where nothing is predefined, and it has to learn everything on its own. The AI we have built into StarCraft II is an example of one of the scripted AIs. Every strategy this AI uses is pre-built and only as clever as a designer that wrote it.
The difficulty levels of this AI are also manually tuned by a game designer. It is this handcrafted approach that we use for all of our games at Blizzard, and this approach, It is also true for the hobbyist community who write customized AI for our games; and there is even an entire tournaments scene built around this style of AI in StarCraft: Brood War.
Now the approach DeepMind is taking is very different. Instead, their AI has to teach itself how to play the game. It has to learn how to extract meaning out of raw image data, and then from those raw features, it has to figure out what the rules of the game even are; and from there it has to piece it all back together, and develop strategies. Nothing like this has been done before in this complex of a game.
Now to get a better sense of what this looks like, let’s watch a video that demonstrates how these AI see the game. On the right hand side of the screen, you will see the regular game view, and on the left you will see a visualization for Deep Learning that we have currently been experimenting with. Now keep in mind that this isn’t a long video, so we are going to play it twice to let you see both sides. So let’s take a look.
Alright, so that looks really cool, but let’s dig into how it actually works. So there are two main ways that we are helping DeepMind accomplish this. The first is by building an interface in the StarCraft II to facilitate reinforcement learning techniques. This is the same example that you heard Oriol talk about using their previous projects. So in this case, the agent is a computer player powered by DeepMind‘s AI, and the environment in StarCraft II.
We then provide a way for this agent to both input actions into the game as well as observe the state of the game, and get feedback on how well it is doing; and this provides a feedback mechanism for an AI to experiment in the environment, and learn from its mistakes.
The second way we are hoping is by analyzing replays. These type of AI start off with absolutely no knowledge of the game. So just like when you first started playing, one of the best ways for you to learn was by watching someone else play, and mimicking what they do; and replays provide an AI the perfect means for this type of learning.
In these replays, the sequence of actions you make… in code, are everything these AIs need to be successful. Everything from Unit micro to strategies, and by using these replays we can build what is effectively a coaching tool for AIs. Now that’s easy to say but it is obviously a lot of challenges that need to be solved to make this happen.
We are working close with DeepMind and together we built a set of tools in the StarCraft II to facilitate this research, and now to make even more progress on this challenge, we are going to be opening up all of these tools to the rest of the research community, and even if you are not an engineer, you can still contribute to this project by simply playing on the ladder.
You can help us generate the replays that can be used to teach these AIs how to play the game. We have also been really impressed on the other hand with the script AIs written in the StarCraft: Brood War community. We’d like to provide them a way to also experiment with AI in StarCraft II.
So to accomplish all this, we are going to be releasing a package we are calling the StarCraft II API. This will provide hooks into our game to facilitate AI development. Our goal for this is to support both cutting-edge deep learning research, as well as custom scripted AIs; and this package will include everything you need to get up and running quickly, and it will also contain a way for multiple of these AIs to play against each other.
We are currently aiming to release this package in the first quarter of next year. Now if you’d like more technical details about this there will be a blog post going up on StarCraft2.com shortly after this panel.
We have even more features planned for this API which we will be talking about in later updates. Now most of you here are probably not AI researchers, but there is still a lot to be excited about. Advancements made here open the door to new game features that weren’t even possible before. For example, you can imagine how having an expert AI at your side would be the perfect coach, and think about how balance testing could work if we could use thousands of these AIs playing against each other.
It is pretty difficult to predict exactly where this will take us, but it has the potential to truly make an impact in games, and the wider world. We are excited at StarCraft II to get the chance to contribute to this effort. So to wrap things up, we are opening up StarCraft II to be the next big AI research platform. To make this happen we are opening up an API early next year to facilitate this research; and we are hoping to bring together both players and researchers to all come help us make this happen.
|BlizzCon 2016 StarCraft II DeepMind Panel Transcript|
|Introduction to DeepMind||StarCraft II and DeepMind|