报告题目：On Convergence of Asynchronous Stochastic Gradient Descent with Unbounded Delay in Non-Convex Learning
报告人：Jia (Kevin) Liu
时间：2018年5月15日（星期二）上午10:00-11：20
地点：逸夫科学馆214会议室
欢迎广大师生参加！
报告摘要：
Asynchronous stochastic gradient descent method (Async-SGD) has become a fundamental building block of many distributed machine learning frameworks. Due to their foundational role in machine learning theory, understanding the convergence performance of Async-SGD has received increasing attention in recent years. To date, however, most of the existing works in this area are restricted to either bounded gradient delays or convex settings. In this talk, we focus on Async-SGD for non-convex optimization problems with unbounded gradient delays. We analyze the convergence performance of standard Async-SGD and an Async-SGD variant with increasing batch size (Async-SGDI) aiming for variance reduction. We prove asymptotic o(1/sqrt{k}) convergence rate for the standard Async-SGD algorithm and o(1/k) for Async-SGDI. Also, we develop a unifying sufficient condition for Async-SGD's convergence that includes two major gradient update delay models in the literature as special cases and implies a new delay model not considered before. Collectively, our results in this paper advance the understanding of the convergence performance of Async-SGD for non-convex learning with unbounded gradient update delay.
报告人简介：
Jia (Kevin) Liu is currently an Assistant Professor in the Dept. of Computer Science at Iowa State University. He received his Ph.D. degree from the Bradley Dept. of Electrical and Computer Engineering at Virginia Tech. Prior to joining Iowa State University, he was a Postdoctoral Researcher and subsequently a Research Assistant Professor in the Dept. of Electrical and Computer Engineering at The Ohio State University. His research areas include theoretical foundations of control and optimization for stochastic networked systems, distributed algorithms design, optimization of cyber-physical systems, Internet-of-things, data analytics infrastructure, and machine learning. Dr. Liu is a senior member of IEEE and a member of ACM. His work has received numerous awards at top venues, including IEEE INFOCOM'16 Best Paper Award, IEEE INFOCOM'13 Best Paper Runner-up Award, IEEE INFOCOM'11 Best Paper Runner-up Award, and IEEE ICC'08 Best Paper Award. He is a recipient of Bell Labs President Gold Award in 2001 and China National Award for Outstanding Ph.D. Students Abroad in 2008. His research has been supported by NSF, AFOSR, AFRL, and ONR.Notably, he is a proud SCUT alumnus . For further information, please visithttp://web.cs.iastate.edu/~
