On May 2nd, the Facebook AI Research (FAIR) official website blog announced the open source of its AI go-go robot ELF OpenGo (including source code and a trained model). The robot was developed based on the ELF platform and defeated 4 worlds. Top 30 Go chess players.
It is understood that this work is mainly inspired by DeepMind's work. Earlier this year, FAIR launched the work, expecting that the Extensible Lightweight Framework (ELF) developed on the basis of its intensive learning research will reproduce the recent results of AlphaGoZero. Its purpose is to create an open source implementation system that can learn Go to reach the level of human professional players or higher levels; at the same time, they also hope to encourage more people to think about this technology through the release of code and models. New applications and research directions.
According to Tian Yuandong, the person in charge of the project, they used 2,000 GPUs to train the model. After about two to three weeks, they got the current version of the Go AI model, which basically exceeded the level of the strong professional player. ELF OpenGo In a match against the world's top 30 four professional chess players (Jin Zhixi, Shen Zhenxi, Park Yongxun, and Cui Zhexi), OpenGo won all games with a score of 14:0; OpenGo uses a single block during the game. GPU search time of 50 seconds per step (search for 80,000 positions per step), and human players can think for any length of time.
In addition, ELF OpenGo is in the same type as LeeLaZero, the strongest go-to-play robot currently available, which uses a default configuration other than ponder, and public weight (192x15,158603eb, Apr. 25, 2018), OpenGo won all 200 games.
Dr. Tian Yuandong’s subsequent publicationAn impression articleIt was mentioned that the OpenGo project "is not designed to be the best Go program, not to defeat anyone." There are three purposes for doing this project:
(1) The AlphaGoZero/AlphaZero algorithm is very interesting. We want to know why it works and how it works. Is it like the publicity that is the universal algorithm of the hundred test lark, isn't it just a heap machine, strong artificial intelligence immediately coming? Or is there any problem or weakness in this algorithm? DeepMind does not open source and does not disclose details. Some parts of the article are not well-written. I wrote blogs before, but it is not always practical without first-hand experience. Therefore, for the purpose of research, we need to reproduce something, first reproduce it, and then have innovation. This is the habit of doing research.
(2) I rewritten the ELF core code earlier this year. In addition, I also added distributed training. I need to find a specific application to test. From this point of view, AlphaGoZero/AlphaZero is the perfect choice, and before that there was DarkForest's code and Go experience, so it doesn't take too much effort to put them together.
(3) Whether it is a general-purpose algorithm or a distributed platform, it can be used to do a lot of other things, not necessarily Go, not necessarily a game. If we look at the ELF OpenGo code, we will find that a large part of it is not related to Go. It is entirely suitable for work in other directions. Go, for us, is just a means to do a good job of algorithms and platforms. At this point, it's worth spending some time to make Go well.
Dr. Tian Yuandong said that with the release of ELF OpenGo today, these three objectives have been achieved. Of course, they hope that they can take this opportunity to promote the ELF platform and PyTorch deep learning framework so that more people can use and improve it.
English blog see:Https://research.fb.com/facebook-open-sources-elf-opengo/
ELF platformIt is an extensible, lightweight and very flexible game research platform, especially for the research of real-time strategy (RTS) games. It was first released in June 2016.
On the C++ side, ELF and C++ threads host multiple games at the same time; on the Python side, ELF can return a batch of game states at once, which makes it very friendly to modern reinforcement learning. In contrast, other platforms (such as OpenAI Gym) simply use a Python interface to encapsulate a single game, which makes the execution of concurrent games (many modern reinforcement learning algorithms have this requirement) more complicated.
ELF has the following characteristics:
End-to-end:ELF provides end-to-end solutions for game research. It provides a miniature real-time strategy game environment, parallel simulations, intuitive APIs, web-based visualizations, and an enhanced learning backend licensed by Pytorch with minimal resource requirements.
ScalableSex:Any game with a C/C++ interface can be inserted into this framework by writing a simple wrapper.
Lightweight:ELF runs very fast and has little overhead. On a MacBook Pro, a simple game based on the RTS engine (MiniRTS) uses the ELF platform and can run single-core 40K frames per second; only 6 CPUs + 1 GPU are used to train MiniRTS from scratch, only 6 day.
flexibility:The pairing between the environment and the participants is very flexible. For example, a single agent and a single environment (such as Vanilla A3C), multiple agents and a single environment (Self-play/MCTS), or a single agent and multiple environments (BatchA3C, GA3C), and the like. Moreover, any game built on top of the RTS engine has full access to its internal representation and dynamics.
With OpenGo's open source, FAIR also made improvements to the ELF platform, upgraded more efficient and friendly APIs, and supported a framework for distributed computing on thousands of computers. As part of PyTorch, ELF can enable researchers to easily experiment with different reinforcement learning ideas and conduct quick and flexible experiments.