 |
Speaker: Chaoyue Liu
Date: Mar 29, 11:45am–12:45pm Abstract: We are now observing an ongoing “AI spring” powered by the emergence and successful implementation of deep neural network models. However, neural networks are often considered as black box models lacking a theoretical and fundamental understanding, leaving deep learning mostly driven by heuristics, not interpretable and not fully reliable. My research focuses on (1) opening the black boxes by discovering fundamental properties of neural networks and deep learning algorithms, and (2) applying these findings to practice and/or theory to push forward the boundary of deep learning. In this talk, I will illustrate that neural networks and deep learning algorithms can be theoretically understood, to a greater extent than you expected, and show how powerful these findings can be when applied. Specifically, in the first part, I will introduce a new and interesting property of neural networks – transition to linearity, and then apply it to optimization theory to obtain fast convergence guarantee of (stochastic) gradient descent on the non-convex loss function of neural networks. In the second part, I will show that the popularly used Nesterov momentum does not provide an expected acceleration when implemented onto SGD, even for linear regression problems. Instead, our new algorithm, MaSS, is provable to provide an acceleration in the stochastic setting, and experimentally outperforms the commonly used algorithms (including Nesterov SGD, and Adam) in real world tasks including non-convex deep learning ones. Finally, I will discuss future directions. Biographical Sketch: Chaoyue Liu is a postdoc at Halicioglu Data Science Institute (HDSI) <https://datascience.ucsd.edu/>, UC San Diego Location and Zoom link: 307 Love, or https://fsu.zoom.us/j/94320318497 |
|
