College students build AI models from scratch, and the ASC22 supercomputer competition challenges them to the limit
The 2022 ASC World University Supercomputer Competition (ASC22) has entered the preliminary stage.More than 300 teams from around the world are challenging an ARTIFICIAL intelligence puzzle called “AI Language Model”.The organizing committee provided 100GB high quality Chinese data set, and required the participating teams to implement a 4.7 billion parameter “source” AI language model based on this data set, so as to stimulate the interest and creativity of the participating college students in natural language processing, and encourage them to launch a challenge to this “crown jewel” of artificial intelligence.With its excellent precision and high intelligence level in application, AI large model has become a hot spot in artificial intelligence research.In the paper jointly published by Professor Feifei Li, the significance of AI large-scale model lies in the emergence and homogeneity. Emergence means that the knowledge implied by large-scale model and inference can bring exciting scientific innovation inspiration, while homogeneity means that a large number of models can provide unified and powerful algorithm support for many application task generalization support.In the past year, there have been a number of outstanding achievements in the field of AI large-scale models, such as “Source 1.0” and “Megatron Turing”.These large models are not only capable of traditional natural language processing tasks, but also capable of writing poems, programming, novels, abstracts and so on, showing broad application prospects in medical, finance, retail, meteorology, journalism, communication, literature and art fields.Although large model has great development potential and application prospect, it also faces the challenge of computing power.The source, for example, used 2,128 accelerators for 16 days of training, which cost a lot of computing power.Therefore, performance optimization of distributed training becomes an important research direction of large-scale model.ASC22 organizing Committee provided 100GB high quality Chinese data set and required the teams to implement a 4.7 billion parameter “source” AI language model based on this data set.However, the organizing committee did not provide a reference code for large-scale model design.This means that the teams need to build the model structure and complete the entire training process from scratch, and design the model training strategy reasonably in order to obtain the best computing performance.As a result, the challenge becomes even more challenging: the pursuit of extreme performance while also meeting the accuracy constraints, which are often the key issues faced by industry professionals in the development of real large models.As Wang Endong, founder of ASC and academician of the Chinese Academy of Engineering, said, with the perfect integration of artificial intelligence and computing power, computing is evolving towards intelligent computing, which may make the next generation of supercomputers into super intelligent computing machines, which not only increase computing performance by an order of magnitude, but also better integrate machine learning and physical modeling.Therefore, ASC22 sets natural language processing, a cutting-edge application combining HIGH performance computing and artificial intelligence, as the competition topic, which will be the perfect testing ground for the teams to compete the fusion ability of AI and supercomputing.Wu Shaohua, an expert on the large-scale AI model competition and chief researcher of Inspur Artificial Intelligence Research Institute, said the competition aims at the direction of distributed training performance optimization, requiring teams to complete 1 billion tokens training on a 100GB data set. Under the condition of meeting accuracy, the faster the performance, the higher the score.The improvement of training performance will directly reduce the training cost of large model, reduce cluster energy consumption, and then reduce carbon emissions.It is hoped that through this competition, the participating teams can form a clear understanding of the cutting-edge research in the field of natural language processing, and through innovative practices, find a universal method to achieve breakthroughs in computing performance.Organized by China and supported by experts and institutions from Asia, Europe and the United States, THE ASC World College Student Supercomputing Competition aims to promote the exchange and training of young supercomputing talents among countries and regions through the platform of the competition, improve the level of supercomputing application and research and development capabilities, give full play to the driving force of supercomputing technology, and promote scientific and industrial innovation.ASC Supercomputer Competition has been held for the 10th time, attracting more than 10,000 college students from all over the world. It is the largest supercomputer competition in the world.For the latest 2022 season, more than 300 teams from around the world have signed up, with those selected in the preliminary round going to the finals at the University of Science and Technology of China in Hefei from May 7 to 11.