After Vowing to Make Pangu World No.1, Yu Chengdong Opens openPangu-2.0-Pro: 505B Parameters, 180B Sparse Activation, 512K Context, First Frontier Model Fully Trained on Ascend NPU

On July 31, Huawei launched openPangu-2.0-Pro as an open-source model featuring 505 billion parameters and 180 billion sparse activations, with a context length of 512K. This model is notable for being the first fully trained on non-NVIDIA hardware, utilizing 34 trillion tokens for training. Huawei provides a fully reproducible chain from training to inference, opening seven components including pretraining and post-training code. Yu Chengdong explained that the choice of 505 billion parameters was influenced by the limited allocation of computing power available to Huawei.