报告题目：On  Convergence of Asynchronous Stochastic Gradient Descent with Unbounded Delay in  Non-Convex Learning

报告人：Jia  (Kevin) Liu

时间：2018年5月15日（星期二）上午10:00-11：20

地点：逸夫科学馆214会议室

欢迎广大师生参加！

报告摘要：

        Asynchronous  stochastic gradient descent method (Async-SGD) has  become a fundamental building block of many distributed machine learning  frameworks. Due to their foundational role in machine learning theory,  understanding the convergence performance of Async-SGD  has received increasing attention in recent years. To date, however, most of the  existing works in this area are restricted to either bounded gradient delays or  convex settings. In this talk, we focus on Async-SGD  for non-convex optimization problems with unbounded gradient delays. We analyze  the convergence performance of standard Async-SGD and  an Async-SGD variant with increasing batch size (Async-SGDI) aiming for variance reduction. We prove  asymptotic o(1/sqrt{k}) convergence rate for the  standard Async-SGD algorithm and o(1/k) for Async-SGDI. Also, we develop a unifying sufficient condition  for Async-SGD's convergence that includes two major  gradient update delay models in the literature as special cases and implies a  new delay model not considered before. Collectively, our results in this paper  advance the understanding of the convergence performance of Async-SGD for non-convex learning with unbounded gradient  update delay.

报告人简介：

         Jia (Kevin) Liu is  currently an Assistant Professor in the Dept. of Computer Science at Iowa State  University. He received his Ph.D. degree from the Bradley Dept. of Electrical  and Computer Engineering at Virginia Tech. Prior to joining Iowa State  University, he was a Postdoctoral Researcher and subsequently a Research  Assistant Professor in the Dept. of Electrical and Computer Engineering at The  Ohio State University. His research areas include theoretical foundations of  control and optimization for stochastic networked systems, distributed  algorithms design, optimization of cyber-physical systems, Internet-of-things,  data analytics infrastructure, and machine learning. Dr. Liu is a senior member  of IEEE and a member of ACM. His work has received numerous awards at top  venues, including IEEE INFOCOM'16 Best Paper Award, IEEE INFOCOM'13 Best Paper  Runner-up Award, IEEE INFOCOM'11 Best Paper Runner-up Award, and IEEE ICC'08  Best Paper Award. He is a recipient of Bell Labs President Gold Award in 2001  and China National Award for Outstanding Ph.D. Students Abroad in 2008. His  research has been supported by NSF, AFOSR, AFRL, and ONR.Notably,  he is a proud SCUT alumnus . For further information,  please visithttp://web.cs.iastate.edu/~jialiu.