Huawei unveils a new Pangu AI model trained on Ascend chips
Huawei has recently launched a new open-source AI model called Pangu Pro MoE 72B, trained on Ascend chips. The company has also pushed off this iconic model to the open-source category, allowing all developers to access its code and capabilities.
Pangu “Pro MoE” 72B (72-billion-parameter) is a hybrid expert model. It uses statistical models to analyze data and symbolic AI, and provides insights into meaning.
The Chinese tech giant has trained the Huawei Pangu Pro MoE 72B AI model with Ascend GPU and NPU chips. Pangu Pro MoE is a sparse model based on MoGE with 72 billion total parameters. 16 billion of these parameters are activated for each token.
Huawei Pangu Pro MoE inference performance can score 1148 tokens/s per card and can further improve to 1528 tokens/s per card when decoding it on Ascend 8001 A2.
It also helps in achieving an excellent cost-to-performance ratio for model inference on the Ascend 3001 Duo. Let’s take a look at the architectural structure of the new Pangu AI model.
Earlier, the company used to implement MoEs (Mixture of Experts) in large language models. This technique is cost-effective for a much larger model and learning capacity.
But the method led to inefficiency as only a small fraction of parameters were used to activate each input token. This often becomes an issue while running the experts on different devices in parallel. Certain ways are used to reduce this issue, but they haven’t eliminated it.
Thus, Huawei introduced the MoGE (Mixture of Grouped Experts), which groups the experts during selection and balances the expert workload better than MoE in nature.
MoGE architectural design maintains a balanced computational load when a model execution begins distribution on multiple devices. It enhanced the overall data transmission, especially for the inference phase. Hence, Pangu Pro MoE 72B can perform better than the previous versions.
(IImage Credits: Huawei)
The post Huawei unveils a new Pangu AI model trained on Ascend chips appeared first on Huawei Central.
