Stabilizing Bi-Level Hyperparameter Optimization using Moreau-Yosida Regularization

ICML 2020

Stabilizing Bi-Level Hyperparameter Optimization using Moreau-Yosida Regularization

Jul 18, 2020
|
36 views
|
Details
This research proposes to use the Moreau-Yosida envelope to stabilize the convergence behavior of bi-level Hyperparameter optimization solvers, and introduces the new algorithm called Moreau-Yosida regularized Hyperparameter Optimization (MY-HPO) algorithm. Theoretical analysis on the correctness of the MY-HPO solution and initial convergence analysis is also provided. Our empirical results show significant improvement in loss values for a fixed computation budget, compared to the state-of-art bi-level HPO solvers. Speakers: Mohak Shah, Sauptik Dhar, Unmesh Kurup

Comments
loading...