标题：Accelerating Deep Neural Network Training with Decentralized Optimization

报告时间：2024年04月22日（星期一）10:00-11:00

报告地点：人民大街校区数学与统计学院105教室

主讲人：袁坤

主办单位：数学与统计学院

报告内容简介：

　　Decentralized optimization algorithms save remarkable communication overheads in distributed deep learning since each node averages locally with neighbors. The network topology connecting all nodes determines communication efficiency and local averaging effectiveness. The key to making decentralized algorithms efficient is to seek sparse topologies that realize effective local averaging with little communication. However, exiting common topologies either suffer expensive per-iteration communication or slow consensus rates. In this talk, we will propose several sparse and effective topologies that endow decentralized algorithms with state-of-the-art balance between communication efficiency and convergence. We will also discuss BlueFog, an open-source python library for straightforward, high-performance implementations of diverse topologies and decentralized algorithms.

主讲人简介：

　　Dr. Kun Yuan is a Researcher at Center for Machine Learning Research (CMLR) in Peking University. He completed his Ph.D. degree at UCLA in 2019, and was a staff algorithm engineer in Alibaba (US) Group between 2019 and 2022. His research focuses on the development of fast, scalable, reliable, and distributed algorithms with applications in large-scale optimization, deep neural network training, federated learning, and Internet of Things. He was the recipient of the 2017 IEEE Signal Processing Society Young Author Best Paper Award, and the 2017 ICCM Distinguished Paper Award.