StarCraft II Learning environment full overview (I)
Overview
In this set of articles , you will learn about Starcraft II Learning Environment project and to train your first Deep Reinforcement Learning agent. You will also discover an overview of DRL and some of the concepts and frameworks to code a ML agent.
Note: The goal is to offer a general overview of how the SC2 environment works, bring out some machine learning concepts to get started.
Why Starcraft II ?
Computer games provide a compelling solution to the issue of evaluating and comparing different learning , control and planning approaches on standard tasks. They present themselves as perfect sandboxes to agents to be trained before being applied to real world problems . Behind this challenges we´re facing a General purpose learning problem,as videogames are systems that are capable of learning how to adapt in the real world .
SC2 is a Real Strategy Game developed by Blizzard after Starcraft I and BloodWar. It is an adversarial game : The goal is to collect resources, build a base, build some units and defeat the opponent that is trying to do the same thing.
Starcraft II presents as a game :
- Vast and diverse action space. There are listed approximately 500 actions as options that can be taken by an agent , and actions available that are displayed by the API . Actions among a combinatorial space of approximately 10e8 possibilities , divided in unit movements , building types and the set of actions evolves as the game and player does . The basic set of actions are approximately 50 times more than Atari games, so it adds a significant scale jump to the choices an agent might take , which leads to think about hierarchical actions, in both macro -strategy and economy- and micro -melee management-.
- Imperfect information game.Full game is a partially observable Markov decision process for the agent -Fog of war configuration-which forces the agent to explore in order to control de map and guess what the opponent is doing.
- Multi-agent problem in high and low level ,as several players compete for influence and resources and each player controls hundred of units which need to collaborate to achieve the goal.
- Sequential problem.The player must take early decisions with consequences that may not be seen until much later in the game. This situation makes the challenge suitable with Reinforcement Learning credit assignment problem.
What you’ll learn
- What is pySC2
- Basics theoretical concepts of Deep Reinforcement Learning to get started
- Basic interaction with API
- Creating and training your Machine Learning agent
- Evaluating how training and evaluating works
- Train a machine learning agent in GCP
What you’ll need
- A computer connected to the internet with Python3 installed
- The library pysc2 installed
- Starcraft II game -no license required-
- keras
- keras-rl
What you need to know
- Basic driving of bash console
- Basic knowledge of python — data structures, lists, functions, loops -
- Basic knowledge of Object-oriented programming -classes, methods, decorators -
- Conceptual knowledge about Neural Networks
Note: the articles install requirements takes a considerable amount of time, so the suggestion is to get ahead in the theory while installation is running
Go to the next step here :
