According to Lei Feng, the work was mainly inspired by DeepMind's work. Earlier this year, FAIR started the work, expecting to conduct intensive study based onDevelopmentThe Extensible Lightweight Framework (ELF) reproduces recent results from AlphaGoZero. Its purpose is to create an open source implementation system that can learn Go to reach the level of human professional players or higher; at the same time, they also hope to encourage more people to think about this technology through the release of code and models. New applications and research directions.
According to Tian Yuandong, the person in charge of this project, they used 2,000 GPUs to train the model. After about two to three weeks, they got the current version of the Go AI model. This model basically surpassed the level of strong professional players. ELF OpenGo In a match against the world's top 30 four professional chess players (Jin Zhixi, Shen Zhenwei, Park Yong Xun, and Cui Zheqi), OpenGo won all games with a score of 14:0; OpenGo used a single block during the game. GPU search time of 50 seconds per step (search for 80,000 positions per step), and human players can think for any length of time.
In addition, ELF OpenGo is one of the most publicly available Go robots, LeelaZero, who uses the default configuration except Ponder and public weights (192x15, 158603eb, Apr. 25, 2018). OpenGo won all 200 games.
Dr. Tian Yuandong subsequently mentioned in a sentimental article published in the episode that the OpenGo project "is not intended to be the best Go program, not to defeat anyone." There are three purposes for doing this project:
(1) The AlphaGoZero/AlphaZero algorithm is very interesting. We want to know why it works and how it works. Is it like the publicity that is the universal algorithm of the hundred test lark, isn't it just a heap machine, strong artificial intelligence immediately Is it coming? Or is there any problem or weakness in this algorithm? DeepMind does not open source and does not disclose details. Some parts of the article are not written clearly. I wrote a blog before,However, there is no practical first-hand experience. Therefore, for the purpose of research, we need to reproduce something, first reproduce it, and then have innovation. This is the habit of doing research.
(2) I rewritten the ELF core code earlier this year. In addition, I also added distributed training. I need to find a specific application to test. From this point of view, AlphaGoZero/AlphaZero is the perfect choice, and before that there was DarkForest's code and Go experience, so it doesn't take too much effort to put them together.
(3) Whether it is a general-purpose algorithm or a distributed platform, it can be used to do a lot of other things, not necessarily Go, not necessarily a game. If we look at the ELF OpenGo code, we will find that a large part of it is not related to Go. It is entirely suitable for work in other directions. Go, for us, is just a means to do a good job of algorithms and platforms. At this point, it's worth spending some time to make Go well.
Dr. Tian Yuandong said that with the release of ELF OpenGo today, these three objectives have been achieved. Of course, they hope that they can take this opportunity to promote the ELF platform and PyTorch deep learning framework so that more people can use and improve it.
See the code at: https://github.com/pytorch/ELF
See the model: https://github.com/pytorch/ELF/releases
English blog see: https://research.fb.com/facebook-open-sources-elf-opengo/
The ELF platform is a scalable, lightweight, and very flexible game research platform, especially for research on real-time strategy (RTS) games. It was first released in June 2016.
On the C++ side, ELF and C++ threads host multiple games at the same time; on the Python side, ELF can return a batch of game states at once, which makes it very friendly to modern reinforcement learning. In contrast, other platforms (such as OpenAI Gym) simply use a Python interface to encapsulate a single game. This situation makes the execution of concurrent games (many modern reinforcement learning algorithms have this requirement) complicated.
ELF has the following characteristics:
End-to-end:ELF provides end-to-end solutions for game research. It provides a miniature real-time strategy game environment, parallel simulations, intuitive APIs, web-based visualizations, and an enhanced learning backend licensed by Pytorch with minimal resource requirements.
ScalableSex:Any game with a C/C++ interface can be inserted into this framework by writing a simple wrapper.
Lightweight:ELF runs very fast and has little overhead. In oneMacBookOn the Pro, a simple game based on the RTS engine (MiniRTS) uses the ELF platform and can run single-core 40K frames per second. It only takes 6 days to train MiniRTS from scratch with only 6 CPUs and 1 GPU.
flexibility:The pairing between the environment and the participants is very flexible. For example, a single agent and a single environment (such as Vanilla A3C), multiple agents and a single environment (Self-play/MCTS), or a single agent and multiple environments (BatchA3C, GA3C), and the like. Moreover, any game built on top of the RTS engine has full access to its internal representation and dynamics.
With OpenGo's open source, FAIR also made improvements to the ELF platform, upgraded more efficient and friendly APIs, and supported a framework for distributed computing on thousands of computers. As part of PyTorch, ELF can enable researchers to easily experiment with different reinforcement learning ideas and conduct quick and flexible experiments.