DPU-The Third Pillar of Computing Power

It is necessary to discuss Nvidia’s DPU (Data Processing Unit, data processor). It can be said that it represents a new direction for the data center. This track has also attracted some start-ups. They might break their hands with giants in segmented application fields. As long as they solve problems of data center infrastructure, such as system performance, throughput, and efficiency.

Nvidia DPU: Exploring the second wave of opportunities in the data center

In the first wave of cloud computing power soaring development, GPU is the king. Various dedicated AI chips for deep learning and other workloads have also ushered in great development. AI will be the key direction of future technological development, such as AI and APP development. We thought Nvidia reached its peak in the data center, he was looking for a second engine of high growth.

The logic behind this is that communication and computing are like the two sides of the data center infrastructure. It’s possible the computing power is constantly increased, but the communication infrastructure cannot keep up. As a result, the overall system performance is limited and it will never be able to realize its true potential. From the perspective of the long-term iteration of the data center, the laying of optical networks, the density of communications, and the construction of ecological facilities with underlying computing power are all in the process of gradual upgrading. With the continuous increase of bandwidth, there will be a greater influx of data. And the processing of data will become more and more complex, including network protocol processing, storage compression, data encryption, etc. These are not areas where CPUs and GPUs are good at. Who should do it? deal with?

The background of DPU is to deal with the exponential increase in the amount and complexity of these data. In the future, it is necessary to move the calculation to a position close to the data. This is an innovation recognized by the industry under a data-centric architecture, rather than a traditional one. The method of moving the data to the calculation location and then processing it.

Product Ideas

AWS calls the “datacenter tax” the “computing resources that will be consumed by connecting to the network data before running a business program”. Considering the data packet processing of a 10G network, it accounts for about half of the computing power of an 8-core CPU. If it is upgraded to a 40G or 100G high-speed network, the overhead in this area will be even greater. Therefore, AWS’s Nitro product boldly runs the virtualization hypervisor originally running on a general-purpose CPU on proprietary hardware. The same idea is also the Alibaba Cloud’s Dragon architecture. The official description mentions that “a containerized product with I/O offload chip acceleration, efficient scheduling, and automated elastic scaling can help users reduce computing costs by 50%.”

From the perspective of the industry’s current development of DPU, as a programmable processor, it runs non-application loads. So server CPU resources can better serve application loads. In a word, for the data center, the DPU achieves efficiency improvement through a more detailed division of labor. For a product that can be Cost-Down, why would there be no market?

Nvidia also took a fancy to this point. The increase in GPU computing power is fierce, but judging from the CPU memory and PCIe bandwidth in the existing x86 system, the overall system performance is still difficult to take off. To this end, Nvidia has launched the first CPU Grace designed for TB-level accelerated computing. And DPU BlueField-3, which is designed for modern ultra-large-scale cloud technology infrastructure. At this point, Nvidia’s data center roadmap has been reborn, with CPU, GPU, and DPU forming three pillars of computing power.

The prototype of DPU is IPU?

Nvidia’s DPU products can be traced back to Mellanox. The acquisition of this company enables Nvidia to quickly make up for its shortcomings in improving the speed of collaboration between servers.

It is worth mentioning that, before being crowned “DPU” by Nvidia, Mellanox’s BlueField product was originally named IPU (I/O Processing Unit), which mainly processes data in the I/O path hardware, through more hardware security features of the company, including agentless micro-segmentation, advanced malware detection, deep packet inspection, and application recognition, realize the speed and flexibility of software-defined solutions, and improve security, accelerate performance, and improve Efficiency.

It is speculated that Huang Renxun’s change of name to DPU should not only inherit the advantages of this hardware solution but also reflect the future trend of data-centric changes. Judging from the current product releases of several generations, DPU has already added obvious Nvidia genes. For example, BlueField-2X adds Ampere-based GPU to DPU on the board, which is called AI-powered DPU by Nvidia. As the industry’s first 400G Ethernet and NDR InfiniBand DPU, BlueField-3 has better network performance. Compared with the previous generation product, it has realized 10 times accelerated computing power, 16 Arm A78 CPU cores, and 4 times the encryption speed.

Security

In addition to computing, storage, and network transmission functions, DPU also shoulders a responsibility that is vital to data services: security. When we put all the equipment, software, and environment in the cloud, the first thing we face is security, and the prevention of security attacks is becoming a new issue. In the past, when the data throughput was not large, it was possible to filter or encrypt traffic through routers, network cards, switches, etc. However, with the popularity of 5G and the Internet of Things, the traditional response methods are no longer applicable. Through DPU, you can respond to data security issues or fight DDoS attacks online. In addition, by adding AI training, it can even identify abnormal traffic and protect the server in a short period of time without human intervention and on duty. This should also be an application scenario for Nvidia’s next AI-based DPU.

According to Nvidia’s forecasts, of the approximately 30 million data center servers delivered each year on the market, 1/3 are used to run software-defined data center stacks. This workload grows much faster than Moore’s Law. The CPU computing power used to run applications in the data center will become less and less.

Summary

In this context, building a data-centric system has become a general trend. DPU, as a large category of the newly developed dedicated processor, is specially designed to meet the needs of computing delay, data security, and resource virtualization under the trend of end-side cloud integration.

Many companies are looking forward to making it one of the three pillars of future computing. When will this day come?

In any case, a new era has arrived.