Dr. Zhang Xia, Chief Strategic Consultant of AWS cloud computing enterprise, said that many restrictions that previously hindered the application of machine learning in the real world began to disappear. Companies around the world, from start-ups to large enterprises, deploying machine learning applications is almost universal priority.

Almost every industry and market segment has begun to apply machine learning to its workload, get more value from data, gain insight and improve business. The golden age of machine learning is also the golden age of AWS.

AWS in the golden age of machine learning

The approximate rate of artificial intelligence has become a deterministic event. More and more enterprises begin to run deep learning, machine learning and other loads. Amazon is the first batch of enterprises. Since the era of e-commerce, Amazon's business of product recommendation, search, logistics and distribution has been integrated into machine learning, and products and businesses such as delivery robot, Amazon echo, Amazon go have been born.

By contrast, machine learning is still a very complex job for enterprises, most enterprises do not have the ability to develop machine learning models independently, AWS and other manufacturers play the role

For example, developers and data scientists must first visualize, transform and preprocess the data, so that the data can become a format that can be used by the algorithm to train the model; from selecting and optimizing the algorithm to adjusting millions of parameters that affect the accuracy of the model, all stages of training the model need a lot of manpower and speculation; deploy the trained data in the application program When modeling, customers need another set of application design and distributed system expertise.

In addition, as the number of data sets and variables increases, the model will become obsolete, and customers must retrain the model again and again to let the model learn and evolve from the new information. All these work need a lot of professional knowledge, and cost a lot of computing power, data storage and time cost. Moreover, because there is no integrated tool for the whole machine learning workflow, the traditional development method of machine learning model is complex, complex and expensive.

The machine learning solution provided by AWS is a three-tier service stack.

The underlying layer is machine learning framework and infrastructure. AWS supports tensorflow, pytorch, Apache mxnet, chainer, gluon, horovod, keras and other machine learning frameworks. In terms of infrastructure, AWS virtual machine provides a variety of instances, while providing off the shelf Amazon machine image (AMI).

Zhang said,

The upper layer of AWS machine learning solution is trained artificial intelligence services, which mainly solve typical problems related to human cognition. For example, computer vision services can identify objects, people, text, scenes, activities and unsafe or inappropriate content in images or videos. Personalized recommendation service can recommend a variety of products and services to consumers from the inventory. Customers can invoke the AI services provided by AWS directly in their applications without paying attention to the machine learning model behind the service.

The middle layer is machine learning service. The main goal is to eliminate the heavy work in the process of machine learning and make it easier to develop high-quality models. It relies on Amazon sagemaker hosting service, which is also the focus of this AWS.

Amazon sagemaker landed in China

Zhang Xia introduced that there are three factors restricting the wide application of AI, which lead to the lack of low-cost, easy-to-use and scalable AI products and services, respectively:

• Lack of talents who master the knowledge of artificial intelligence;

• It is difficult to build and expand AI technology products;

• The deployment of AI application in production and operation is time-consuming and costly.

Amazon sagemaker is to eliminate the heavy work of machine learning steps. On May 12, AWS announced that Amazon sagemaker was officially launched in AWS China (Ningxia) region operated by cloud data and AWS China (Beijing) region operated by halo new network.

Through the preset notebook, common algorithms for Pb level dataset optimization, and automatic model tuning, Amazon sagemaker reduces the difficulty of model building and training. In addition, Amazon sagemaker simplifies and accelerates the model training process, and can train models and run reasoning by automatically providing and managing infrastructure.

At the same time, AWS recently announced a number of important features and advanced features that make it easier for customers to build, train, tune, and deploy machine learning models. These functions include:

• Integrated development environment (IDE) for machine learning: Amazon sagemaker studio gathers all components for machine learning. Developers can view and organize source code, dependencies, documents and other application assets in Amazon sagemaker studio Make building, training, interpreting, checking, monitoring, debugging, and running machine learning models easier and faster.

• Elastic Notebook: Amazon sagemaker notebook provides one click enabled jupyter notebook, which has the second level elastic computing improvement ability, so that developers can easily increase or reduce the computing power required by notebook (including GPU acceleration), which automatically occurs in the background without interrupting developers' work. Amazon sagemaker notebook can also automatically copy specific environment and library dependencies to realize one click sharing of notebook.

• Experimental management: Amazon SageMaker Experiments can help developers organize and track iterations of machine learning models. Amazon SageMaker Experiments automatically capture input parameters, configurations, and results and store them as

• Debugging and analysis: Amazon sagemaker debugger is used to debug and analyze model training, improve accuracy, reduce training time, and let developers better understand the model. Using Amazon sagemaker debugger, the model trained in Amazon sagemaker will automatically send out the collected key indicators. Amazon sagemaker debugger can also help developers understand how the model works and take the first step towards the interpretability of neural networks.

• Automatic model building: Amazon sagemaker autopilot is the first automated machine learning feature in the industry that allows developers to maintain control and visibility over their models. Amazon sagemaker autopilot will automatically check the original data, apply the feature processor, select the best algorithm set, train multiple models, tune them, track their performance, and then rank the models according to their performance. Developers can select the best model for the application scenario, and consider multiple candidate models with different optimization factors.

• Concept drift detection: Amazon sagemaker model monitor allows developers to detect and correct concept drift. Developers can use Amazon sagemaker model monitor's out of the box functionality to detect drift, or write their own rules for Amazon sagemaker model monitor to monitor. Amazon sagemaker model monitor makes it easier for developers to adjust training data or algorithms to solve concept drift problems.

According to IDC report, China's AI market has become the second largest single AI market in the world, and the market scale is still growing rapidly. At present, 40% of enterprise digital transformation projects will use artificial intelligence, which will become an indispensable part of all business departments, promote large-scale innovation and realize huge business value.