面向大模型的智算网络发展研究

doi:10.11959/j.issn.1000-0801.2024147

Telecommunications Science ›› 2024, Vol. 40 ›› Issue (6): 137-145.doi: 10.11959/j.issn.1000-0801.2024147

Previous Articles Next Articles

Research on the development of intelligent computing network for large models

Liang GUO¹^,², Shaopeng WANG¹(), Wei QUAN², Jie LI¹

^1.Cloud Computing and Big Data Research Institute, China Academy of Information and Communications Technology, Beijing 100191, China
^2.School of Electronic and Information Engineering, Beijing Jiaotong University, Beijing 100044, China

Received:2024-04-03 Revised:2024-05-14 Online:2024-06-20 Published:2024-07-11
Contact: Shaopeng WANG E-mail:wangshaopeng@caict.ac.cn
Supported by:
The National Science and Technology Major Project(2021ZD0113003)

Abstract

Abstract:

In recent years, the world has entered a period of vigorous development in intelligent computing. As deep learning models with huge parameters and complex structures, large model training requires fast synchronization of training parameters between multiple cards and servers, which imposes higher requirements on the bandwidth, latency, reliability, scalability and security of datacenter networks. The requirements and related key technologies of intelligent computing networks for large model training were studied, and the standard specifications, academic research, and case practices of intelligent computing networks were analyzed, in order to promote the development of intelligent computing networks.

Key words: large model, intelligent computing center, network technology

CLC Number:

TP393

Liang GUO, Shaopeng WANG, Wei QUAN, Jie LI. Research on the development of intelligent computing network for large models[J]. Telecommunications Science, 2024, 40(6): 137-145.

Figures/Tables 6

References 20

1	CONGDON P, MARKS R. IEEE 802 Nendica report: the lossless network for data centers[R]. 2018.
2	GUO L, CONGDON P. IEEE 802 Nendica report: intelligent lossless data center networks[R]. 2021.
3	孙黎阳, 温小振, 郭亮. 总线级数据中心网络关键技术研究[J]. 中国电信业, 2022(10): 75-77.
	SUN L Y, WEN X Z, GUO L. Research on key technologies of bus-level data center network[J]. China Telecommunications Trade, 2022(10): 75-77.
4	JIANG Z H, LIN H B, ZHONG Y M, et al. MegaScale: scaling large language model training to more than 10 000 GPUs[J]. arXiv preprint, 2024: arXiv: .
5	ZHANG Y W, KUMAR G, DUKKIPATI N, et al. Aequitas: admission control for performance-critical RPCs in datacenters[C]// Proceeding of the 2022 ACM SIGCOMM'22 Conference, New York: ACM, 2022.
6	NAMYAR P, SUPITTAYAPORNPONG S, ZHANG M Y, et al. A throughput-centric view of the performance of datacenter topologies[C]//Proceedings of the 2021 ACM SIGCOMM 2021 Conference. New York: ACM, 2021.
7	AL-FARES M, LOUKISSAS A, VAHDAT A. A scalable, commodity data center network architecture[C]//Proceedings of the ACM SIGCOMM 2008 Conference on Data Communication. New York: ACM, 2008: 63-74.
8	KIM J, DALLY W J, SCOTT S, et al. Technology-driven, highly-scalable dragonfly topology[C]//Proceedings of the 2008 International Symposium on Computer Architecture. Piscataway: IEEE Press, 2008: 77-88.
9	唐雄燕, 魏步征, 沈世奎, 等. 智算数据中心光电交换技术综述[J]. 光通信研究, 2024(4): 1-17.
	TANG X Y, WEI B Z, SHEN S K, et al. Overview of optoelectronic switching technology in intelligent computing data centers[J]. Study on Optical Communications, 2024(4): 1-17.
10	何宝宏, 郭亮, 王少鹏. 星河AI网络白皮书[R]. 2023.
	HE B H, GUO L, WANG S P. Star river AI network white paper[R]. 2023.
11	王少鹏, 郑常奎, 芦帅, 等. 数据中心无损网络关键技术研究[J]. 信息通信技术与政策, 2021(10): 68-74.
	WANG S P, ZHENG C K, LU S, et al. Research on key technologies of data center lossless network[J]. Information and Communications Technology and Policy, 2021(10): 68-74.
12	黄云皓. 大模型时代的“非大模型”机会: 智算中心以太网[R]. 2023.
	HUANG Y H. Opportunities for “non large models” in the era of large models: Ethernet in intelligent computing centers[R].2023
13	郭亮, 李洁, 高峰. 数据中心智能无损网络白皮书[R]. 2021.
	GUO L, LI J, GAO F. White paper on intelligent lossless networking in data centers[R]. 2021.
14	MIAO Y L R, LIU H H. HPCC: high precision congestion control[C]//Proceeding of the 2019 ACM SIGCOMM'19 Conference. New York: ACM, 2019.
15	ZHANG Z, ZHENG S, WANG Y D, et al. MiCS: near-linear scaling for training gigantic model on public cloud[J]. ArXiv e-Prints, 2022: arXiv: .
16	SHUKLA D, SIVATHANU M, VISWANATHA S, et al. Singularity: planet-scale, preemptive and elastic scheduling of AI workloads[J]. arXiv preprint, 2022, arXiv: .
17	张宏科, 于成晓, 权伟, 等. 融算网络体系基础研究[J]. 电子学报, 2022, 50(12): 2928-2934.
	ZHANG H K, YU C X, QUAN W, et al. Fundamental research on computing integration networking[J]. Acta Electronica Sinica, 2022, 50(12): 2928-2934.
18	胡玉姣, 贾庆民, 孙庆爽, 等. 融智算力网络及其功能架构[J]. 计算机科学, 2022, 49(9): 249-259.
	HU Y J, JIA Q M, SUN Q S, et al. Functional architecture to intelligent computing power network[J]. Computer Science, 2022, 49(9): 249-259.
19	腾讯云计算(北京)有限责任公司, 中国信息通信研究院云计算与大数据研究所. 智算赋能算网新应用白皮书[R].2023.
	Tencent Cloud Computing (Beijing) Co., Ltd., Cloud Computing and Big Data Research Institute of the China Academy of Information and Communication Technology. White paper on new applications of intelligent computing empowerment network[R]. 2023.
20	李兆彤, 史磊, 周磊. 智算中心网络架构白皮书[R]. 2023.
	LI Z T, SHI L, ZHOU L. White paper on network architecture of intelligent computing center[R]. 2023.

Metrics

Recommended 0

No Suggested Reading articles found!

[1]	Xiaodong DUAN, Jieyu LI, Weiqiang CHENG, Han LI, Ruixue WANG, Haojie WANG. Challenges and key technologies of new Ethernet for intelligent computing center [J]. Telecommunications Science, 2024, 40(6): 146-159.
[2]	Kexin FAN, Lirong AN, Qinyu ZHANG. Research on service-driven network technologies for space-air-ground-sea integration [J]. Telecommunications Science, 2024, 40(6): 25-37.
[3]	Hucheng Wang,Hui Xu,Zhimi Cheng,Ke Wang. Current Research and Development Trend of 5G Network Technologies [J]. Telecommunications Science, 2015, 31(9): 149-155.
[4]	Xiaojuan Hu,Huiling Zhao. Research on the Virtualized Core Network [J]. Telecommunications Science, 2014, 30(8): 42-46.

Research on the development of intelligent computing network for large models

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 6

References 20

Related Articles 4

Metrics

Recommended 0