ML/Google Distinguished Lecture Series-Machine Learning Department - Carnegie Mellon University

Machine Learning Special Seminars


Spring 2017 Seminar
Date: March 31, 2017
Time: 12:00 PM - 00:00 AM
Location: 1507 Newell-Simon Hall
Speaker: Elad Hazan Professor, Princeton University
Title: Second-order Optimization for Machine Learning in Linear Time
Abstract: Stochastic gradient-based methods are the state-of-the-art in large-scale machine learning optimization due to their extremely efficient per-iteration computational cost. Second-order methods, that use the second derivative of the optimization objective, are known to enable faster convergence. However, the latter have been much less explored due to the high cost of computing the second-order information. We will present a second-order stochastic method for (convex and non-convex) optimization problems arising in machine learning that match the per-iteration cost of gradient descent, yet enjoy convergence properties of second-order optimization. Joint work with Naman Agarwal and Brian Bullins (ICML '16), and Agarwal, Bullins, Allen-Zhu and Ma (STOC '17)