报告题目:Averaged Method of Multipliers for Bi-Level Optimization without Lower-Level Strong Convexity
报 告人:尧伟博士
报告时间:2023年 3月 12 日(星期日)14:00-14:45
报告地点:四号楼4318会议室
邀 请人:潘少华
欢迎广大师生前往!
数学学院
2023年 3 月 8日
报告摘要:Gradient methods have become mainstream techniques for Bi-Level Optimization (BLO) in learning fields. The validity of existing works heavily rely on either a restrictive Lower- Level Strong Convexity (LLSC) condition or on solving a series of approximation subproblems with high accuracy or both. In this work, by averaging the upper and lower level objectives, we propose a single loop Bi-level Averaged Method of Multipliers (sl-BAMM) for BLO that is simple yet efficient for large-scale BLO and gets rid of the limited LLSC restriction. We further provide non-asymptotic convergence analysis of sl-BAMM towards KKT stationary points, and the comparative advantage of our analysis lies in the absence of strong gradient boundedness assumption, which is always required by others. Thus our theory safely captures a wider variety of applications in deep learning, especially where the upper-level objective is quadratic w.r.t. the lower-level variable. Experimental results demonstrate the superiority of our method.
专家简介:博士毕业于香港中文大学,任职南方科技大学数学系和深圳国家应用数学中心研究助理教授,主要研究方向包括双层规划算法和理论,及其在机器学习和理论经济学上的应用。代表性论文发表在SIAM J Optim、Journal of Convex Analysis、Calculus of Variations and Partial Differential Equations、Journal of Differential Equations等运筹优化、偏微分方程领域的国际著名期刊上。