With the advent of the digital age, IT systems have become an integral part of the normal functioning of human society. In the near future, technologies such as smart manufacturing, 5G and artificial intelligence will become important engines for the development of productivity, and human society will face an unprecedented wave of digitalization. As the platform and carrier for the operation of IT systems, IT infrastructure is the cornerstone of digitalization. In this wave of digitization, companies must actively embrace cloud computing technology and adopt an IT infrastructure that is in line with technological trends and future-oriented to gain opportunities in future competition.
First, the cloud computing has been judged by the trend of development for more than ten years.
Cloud computing technology began in AWS, the first public cloud service S3 launched in 2006, and today, in 2019, some patterns and trends begin to become clear:
First, private clouds remain the preferred IT infrastructure for large and medium-sized businesses and for a number of sub-sectors such as government, finance, healthcare, education, energy, and manufacturing. As public cloud vendors continue to roll out their public cloud deployment plans for privatization of government and enterprise customers, such as AWS Outposts, Azure Stack, Google Anthos, and private cloud/proprietary cloud deployment solutions such as Alibaba Cloud and Tencent Cloud in China. , "If the private cloud will gradually disappear with the development of the public cloud", the proposition has been denied by the public cloud vendors themselves. It turns out that private clouds will persist for a long time and will be symbiotic with the public cloud and become an important part of the enterprise's IT infrastructure.
Second, the public cloud continues to grow rapidly and is becoming a major provider of enterprise IT infrastructure. In 2018, Q3 cloud hardware expenditure accounted for 50.9% of total IT revenue. In 2018, China's private cloud infrastructure spending was $3.8 billion, and public cloud infrastructure spending reached $8.2 billion (source: IDC). As a result, public clouds have become the primary provider of IT infrastructure. Especially for SMEs, their IT infrastructure may be built entirely on public clouds. At the same time, some large enterprises in the technology-leading industry, such as the Internet, finance, manufacturing, etc., have also begun to use the public cloud to explore a hybrid cloud architecture that combines the advantages of public and private clouds.
At the same time, the competition in the public cloud market is extremely fierce. In the end, only a few players with very strong technology and capital will enter the oligopolistic market. On the one hand, the products and services provided by public cloud vendors are essentially the same, all of which are IT infrastructure resources and software services on them. On the other hand, each manufacturer strives to improve its products, enrich its product lines, and make features to obtain Competitive advantage, attracting incremental users and avoiding the loss of existing users. Therefore, the product function matrix provided by the public cloud in the end is basically the same, but there are differences in the features, regional coverage, and user experience. As users become more dependent on public cloud product services, service switching and migration between public clouds will become more and more difficult, and there is an invisible gap between cloud and cloud. Of course, the entry of the public cloud into the oligopoly phase also means that the list of public cloud providers will remain relatively stable over the long term, which means that the cost of adaptation for all public cloud APIs will become manageable and feasible.
Another trend that cannot be ignored is that Kubernetes has become the de facto standard for container orchestration, gradually becoming the standard environment for cloud deployment and operation. With Kubernetes' gradual improvement of storage and network support, not only can stateless services be deployed and run on Kubernetes, but stateful data storage services can also run on Kubernetes. At the same time, based on Kubernetes, a prosperous and powerful open source software ecosystem and a complete tool chain have been developed, such as the automatic deployment of Helm implementation software suite, Operator implementation of software automation, and lstio providing microservice RPC communication governance architecture. Knative provides a serverless runtime framework and more. It is foreseeable that Kubernetes will become the standard runtime environment for distributed applications in the future, becoming the “Linux” of the distributed application era. On top of Kubernetes, a software ecosystem dominated by open source software will be built, including not only application software, but also various PaaS middleware, such as message middleware, various open source databases, development frameworks, AI training frameworks, etc. The vision of "open source rules the world." Based on this trend, major public cloud vendors have launched their own Kubernetes solutions, allowing native Kubernetes to run more efficiently on their own cloud platforms.
Second, the determination and uncertainty of the future IT infrastructure of the enterprise
Based on these facts and trends, we can imagine that the future enterprise IT infrastructure will be like this:
First, the hybrid cloud architecture is the best choice for the enterprise.
In the future, enterprise IT infrastructure solutions, public and private clouds are no longer an option, but a two-part component of a complete IT infrastructure. On the one hand, companies may have their own private clouds, but there are also some companies that run entirely on public clouds. On the other hand, companies must use public clouds, and the public cloud resources they purchase will be part of their private IT infrastructure.
Second, Kubernetes will be the standard operating environment for enterprise cloud native applications.
Just as enterprises today run enterprise applications in Linux, future enterprise services will be biochemical, distributed, and run in Kubernetes. Enterprises will have several Kubernetes clusters running different applications, distributed across different infrastructures, some running on local IDCs, some running on private clouds, and some running on public clouds.
The above two points are generally accepted and certain conclusions, but there are many other issues that currently have no definitive conclusions, such as:
1. Although the use of public cloud is an inevitable choice for enterprises, enterprises will choose whether to use multiple public clouds or a single public cloud. There are many reasons for adopting a multi-public cloud solution, and the benefits are obvious, such as avoiding vendor lock-in, improving bargaining power, and obtaining richer feature sets and geographic choices. At the same time, the unified management of using multiple public cloud resources is difficult, and the problem of high cost of service switching and migration between clouds hinders users from selecting multiple public clouds.
2. Although cloud computing technology has been developed for more than a decade, there is still a large percentage of enterprises whose local IT infrastructure is not clouded, neither through private cloud management nor even using virtualization technology. While future cloud native applications will run in Kubernetes' container environment, there are many traditional applications that are not containerized. Moreover, to be realistic, for most companies, perhaps for a long time in the future, it is still based on the traditional application of non-cloud. Therefore, the enterprise's future IT infrastructure cannot simply assume that all of the Kubernetes are run in a normalized manner, but instead should provide these traditional applications with the virtual machines or bare-metal environments required to run them. Does this kind of enterprise cloud transformation process still need to go through a private cloud-hybrid cloud-cloudy long path, and then deploy a private cloud to realize the cloudization of the local IT infrastructure?
3. On the one hand, with business development and industry-driven, enterprises' requirements for IT infrastructure will become more and more demanding in terms of scale, efficiency and stability. Agile development and DevOps will become standard for the enterprise. On the other hand, as technology evolves, enterprise IT infrastructure will become more complex and unmanageable. Enterprise IT resources will not only be physical servers, but also virtual machines and containers. In addition to x86, there will be minicomputers, ARMs, and even heterogeneous computing resources such as GPUs, FPGAs, and TPUs. There are also a variety of technology options for networking and storage. At the same time, as of today, only mainstream public cloud providers offer thousands of cloud products and services in more than 500 Availability Zones in more than 200 locations around the world. A global scale IT infrastructure is at your fingertips as long as the business is willing. How do enterprise IT staff address the challenges of IT infrastructure in terms of scale, efficiency, and complexity?
4. Even if the future enterprise IT infrastructure will converge to run entirely on Kubernetes, a single Kubernetes cluster will often be used for a single specific purpose, such as a specific departmental test or production cluster, and multiple Kubernetes clusters within the enterprise are normal. Managing multiple Kubernetes clusters, especially multiple Kubernetes clusters deployed in a cloudy environment, remains a challenge. Although Kubernetes shields the underlying infrastructure differences and provides a consistent interface and runtime environment, Kubernetes does not have uniform management interfaces and network storage schemes for public clouds and local IDCs. When creating, expanding, and adjusting Kubernetes clusters. Still facing the problem of docking multiple vendor interfaces. At the same time, there is no communication between the Kubernetes clusters distributed on multiple public clouds. Not only the control information cannot be synchronized, but the data link layers are isolated from each other and are isolated islands. Therefore, linkage between multiple clusters cannot be achieved, and switching and collaboration between clusters cannot be achieved. The Kubernetes cluster solution in a cloudy environment remains to be explored.
5. With the improvement of the Kubernetes ecosystem, users will have two options for using PaaS services on public clouds: using the PaaS service provided by the public cloud or the cloud-based open source PaaS service based on Kubernetes. The former is highly productized, easier to use, and commercially supported. However, there are also problems that are locked by commercial products, difficult to switch, and expensive to use. Using the latter requires some control over open source software, but it is cheap (the cloud hosting fee), based on open source technology, with strong community support, open and flexible architecture and easy to expand.
Third, Converged Cloud (Unified IaaS), future-oriented IT infrastructure choice
For the above deterministic and uncertain questions, our answer is that the best choice for future IT infrastructure management is Unified IaaS. As the name implies, the so-called converged cloud is a converged IaaS platform that integrates and manages all IT infrastructures distributed in a cloudy environment (local IDC, private cloud and public cloud) to build a “cloud on the cloud”. Converged clouds are essentially private clouds, but the scope of managed IT resources is no longer limited to local IDCs, but also includes IaaS resources that companies purchase on public clouds. For enterprises with pure public cloud architecture, the integration of cloud management is completely public cloud resources purchased by enterprises. The difference between the converged cloud and the traditional cloud platform is not the difference in the scope of management resources, but in the development trend and problems of the above enterprise IT. In terms of design concept, the converged cloud and the traditional cloud platform have the following differences:
First, the converged cloud is geared towards a cloudy environment.
In a converged cloud deployment scenario, the enterprise user IT infrastructure includes not only the portion of the local IDC that is deployed, but also the portion of the user's purchase in the public cloud. Converged Cloud manages all of your enterprise's IT infrastructure through a single platform. The first is the unification and integration of the management plane to achieve unified API access of private and public cloud resources, not only to achieve resource management, but also to unify bills and unify resource management. Getting users to call across the cloud is as easy as using a cloud platform. Secondly, the data plane is opened. Through the integration with the cross-cloud network solution, the control plane and the data platform are coordinated to achieve the inter-cloud inter-network communication of the entire platform. In addition, Converged Cloud will also provide tools for cross-cloud data migration, enabling users to implement application migration across the cloud. In short, the goal of converging the cloud is to bridge the gap between the cloud and the cloud, reduce the cost of cross-cloud switching and migration, and make cloudy deployment easier.
Second, converged the cloud to achieve a comprehensive cloud of the company's overall heterogeneous IT infrastructure.
Converged Cloud not only manages clouded private and public cloud resources, but also built-in bare metal clouds to manage bare metal, virtualization technologies such as KVM and VMware ESXi, and private cloud technologies for computing resources such as ARM and GPU. For enterprises that have not yet deployed a private cloud, deploy a converged cloud to realize private cloudization of the enterprise's private IT infrastructure in one step, and realize cloud management of bare metal, KVM, VMware ESXi, GPU, etc., without introducing additional private The cloud solution reduces the implementation cost and management complexity of the cloud on the enterprise.
Third, intelligence will be the core feature of the Converged Cloud.
On the one hand, the converged cloud optimizes the scheduling strategy of IT resource allocation, finds idle and wasted IT resources, and improves the utilization of IT resources. On the other hand, predict resource requirements and discover system failures in advance to ensure smooth operation and expansion of the system. Data and algorithms make IT infrastructure smarter, helping enterprise IT to harness the challenges of scale, efficiency, and complexity in future IT infrastructure.
Fourth, the fusion cloud is oriented to Kubernetes.
The fusion cloud on the one hand achieves the unification and integration of Kubernetes' underlying infrastructure in a cloudy environment. One is to provide Kubernetes with a unified IaaS interface in a cloudy environment through a unified API to facilitate the deployment of Kubernetes environments across the cloud. The second is to open the intranet across the cloud Kubernetes in the data plane to achieve cross-cloud communication. On the other hand, it directly provides a unified Kubernetes cluster management control API and cluster information synchronization mechanism to achieve unified management of cross-cluster Kubernetes, and achieve synchronization and unification of accounts, permissions, and configurations across Kubernetes clusters.
Finally, the Converged Cloud embraces open source technologies.
The history of software development shows that the future of PaaS is open source. Suppliers are unable to meet all PaaS needs of corporate customers on their own. As a result, Converged Cloud focuses on the unified management of enterprise, distributed network, and storage IaaS resources in local IDCs and public clouds, providing a reliable underlying infrastructure for cloudy Kubernetes, and software and application requirements on Kubernetes rely on the open source ecosystem. solution. Converged cloud users' demand for PaaS is delivered to users through the Kubernetes application market by integrating open source PaaS applications. This aspect reduces the technical threshold for users to use open source PaaS, and on the other hand relies on a powerful open source community to provide users with open and flexible software products to avoid the lock of private PaaS software.
Based on the above assumptions, the architecture of the converged cloud is as follows.
Downward: Converged cloud unified management of cloudy infrastructure, mainly to achieve unified management of IaaS resources such as computing, network, and storage in a cloudy environment. For the unclouded resources of local IDC, mainly bare metal, KVM virtual machine (Libvirt), VMware ESXi virtual machine (vSphere), cloud management through built-in private cloud solution. For private and public cloud resources, unified management through the API.
Upward: Converged Cloud provides a complete operating system runtime environment for traditional applications through virtual machines, bare metal, etc. On the other hand, it provides Kubernetes with a cloudy running environment and unified management of cloudy Kubernetes. On top of Kubernetes, the container runtime environment for cloud native applications is provided. At the same time, PaaS middleware services are provided based on Kubernetes and open source components.
In summary, the converged cloud manages the cloudy IaaS resources in a unified manner; provides cloudy support for Kubernetes, meets the enterprise PaaS needs through the open source ecosystem; and other users' needs can be obtained by accessing the public cloud's native services, thus fully satisfying the future enterprise-to-IT foundation. Multi-level needs of the facility.
With the spread of big data and artificial intelligence technology, the arrival of the 5G era, IT infrastructure has become more and more important, becoming the digital transformation of enterprises, and fully embracing the cornerstone of the digital age. Based on the cloudy trend of enterprise IT architecture, the convergence cloud came into being. Converged Cloud is a future-oriented enterprise IT infrastructure management cloud platform designed for the future of IT infrastructure issues and will help companies meet the challenges of the upcoming digital transformation.
about the author:
Qiu Jian, Yunlian Wanxi CTO, Tsinghua Benshuo, the ashes-level code farmer, former US Mission Cloud Architect and technical core, led the team to complete the earliest technology selection, architecture design and code delivery of Meituan.