Huawei Unveils Peerium AI Architecture with High-Speed Interconnect System

A groundbreaking innovation in artificial intelligence (AI) has emerged from China, where engineers have designed a new computer architecture that can potentially combine up to a million AI chips into a single powerful system.

This architecture, known as Peerium, is the brainchild of Huawei, a leading technology company based in China. In contrast, other notable AI systems, such as Elon Musk's Colossus cluster at SpaceXAI, currently rely on separate servers and networks to facilitate communication between AI chips, which can be cumbersome and inefficient.

Peerium, unveiled by Huawei at its annual CONNECT conference last September 17, boasts a high-speed interconnect system that enables seamless communication between chips across multiple racks. This allows for the efficient exchange of data and workload distribution, even as the number of chips scales into hundreds of thousands.

The potential of Peerium is vast, with representatives from Huawei stating that computers powered by this architecture could train models with up to 40 trillion parameters. For context, competing hardware, such as NVIDIA's GB200 NVL72 and Cerebras' CS-3, have only reported training at the trillion-parameter scale.

According to Liao Heng, chief scientist for Huawei's chip design arm HiSilicon, China is poised to become a hub for AI innovation in the next two years. He forecasts that approximately 6 or 7 frontier AI labs will emerge in the country, all working towards developing foundation language models with sizes ranging from 10 to 40 trillion parameters.

Huawei's Peerium architecture relies on three key components to optimize its performance: nested parallelism, unified memory addressing, and peer-to-peer interconnects.

Nested parallelism allows workloads to be divided into smaller tasks that can be executed in parallel across multiple chips, similar to how a company organizes departments and teams to achieve a common goal.

Unified memory addressing enables processors to access memory from other servers, rather than being limited to their own server's memory. This allows for more efficient use of computing resources and enables the coordination of larger numbers of chips to handle complex AI workloads.

Peer-to-peer interconnects facilitate direct communication between components, eliminating the need for intermediaries and further enhancing efficiency.

The challenge of scaling up artificial intelligence (AI) processing lies in connecting vast numbers of individual processors to work together seamlessly as a single computer. This hurdle is due to limited inter-rack communications bandwidth, which restricts the ability to synchronize and manage processor activity.

To overcome this limitation, Huawei has developed its own proprietary interconnect technology called UnifiedBus. The company claims that UnifiedBus can achieve impressive inter-rack communications bandwidth of up to 800GB per second. This innovation is a crucial component in making AI processing more efficient and scalable.

Huawei's new architecture, dubbed SuperCluster, relies on Peerium as the underlying interconnect fabric. When paired with the Atlas 950 SuperPoD computing system, which comprises over 8,000 Huawei Ascend 950DT AI accelerator chips, the company is creating a powerful platform for AI processing. The integration of UnifiedBus and Peerium enables processors to work together in perfect harmony.

Testing has begun on these high-performance systems, with initial results indicating that they can utilize their available computing power more efficiently than traditional AI computing clusters. This achievement paves the way for significant advancements in AI training and inference capabilities.

Huawei is gearing up to supply its SuperPoDs by early next year at the latest, marking a crucial step towards establishing its AI computing infrastructure. These modular units will eventually be linked together in larger systems known as SuperClusters, which are based on the Peerium architecture.

The first large-scale deployment of a SuperCluster with 256,000 computing cards is currently underway and being thoroughly tested. This massive system has significant implications for models that require substantial computational resources to operate efficiently, such as those used in natural language processing tasks like predicting the next word or token in a sequence. By leveraging this advanced architecture, Huawei aims to support the development of complex AI applications without drastically increasing costs associated with hardware and infrastructure.