【学术报告】Distributed Systems for Decentralized AI: Last Decade and Beyond

发布时间:2022-11-05 
报告题目:Distributed Systems for Decentralized AI: Last Decade and Beyond
报告人:苏黎世联邦理工 张策教授
时间:2022年11月09日(周三),下午 04:00-05:00
地点:腾讯会议 352-460-473
联系人:徐跃东 老师

 

摘要:Today’s machine learning models are trained dominantly in a centralized environment such as data centers. Such a strong dependency on closely coupled interconnections is causing problems not only with the cost of infrastructure, contributing to the staggering cost of model training (not uncommon to be in the million-dollar regime!) but also with the transparency and openness of today’s machine learning ecosystem. While a decentralized paradigm has been successfully applied to various other areas such as SETI@HOME and Folding@HOME, their adoption to machine learning comes with great challenges — today’s optimization algorithms that are designed for fast networks are inherently communication heavy. To bring AI into a decentralized future, we need to fundamentally rethink the training algorithm, system design, and hardware accelerations, all together.
 

In this talk, I will provide a personal reflection on our efforts in decentralized learning over the last decade, and put them into the context of many fascinating work that has been conducted by the community during the same period of time. I will also discuss my personal view on the future of decentralized learning and a “cry for help” with many challenges that we are still struggling with.

 

简历:Prof. Ce Zhang is an Assistant Professor in Computer Science at ETH Zurich. The mission of his research is to make machine learning techniques widely accessible — while being cost-efficient and trustworthy — to everyone who wants to use them to make our world a better place. He believes in a system approach to enabling this goal, and his current research focuses on building next-generation machine learning platforms and systems that are data-centric, human-centric, and declaratively scalable. Before joining ETH, Ce finished his PhD at the University of Wisconsin- Madison and spent another year as a postdoctoral researcher at Stanford, both advised by Christopher Ré. His work has received recognitions such as the SIGMOD Best Paper Award, SIGMOD Research Highlight Award, Google Focused Research Award, an ERC Starting Grant, and has been featured and reported by Science, Nature, the Communications of the ACM, and various media outlets such as Atlantic, WIRED, Quanta Magazine, etc.