 |
Speaker: Shangqian Gao
Date: Apr 3, 11:45am–12:45pm Abstract: Recently, Machine Learning and Artificial Intelligence have achieved significant success in various domains, such as playing GO, ChatGPT, recommendation systems, autonomous driving, etc. However, the model architectures behind these accomplishments have grown increasingly larger, posing challenges in computation, storage, and memory costs. Motivated by the need for efficiency, numerous researchers have focused on reducing model size. One promising approach is to remove redundant model structures, which can achieve acceleration without requiring additional post-processing steps. In this presentation, we introduce new methods for model compression from the perspective of differentiable pruning. In the first part of this talk, we will discuss methods for enabling efficient discrete optimization of model structures for Convolutional Neural Networks (CNNs). Additionally, we incorporate model weights into the optimization process using the partial regularization technique. By jointly optimizing model weights and structures, we eliminate the need for pre-trained models, allowing for the training of a sparse model within the large model from scratch. In the second part of the presentation, we delve into efficient machine learning methods for reducing the number of parameters in transformers, like grounding-based vision and language models and Large Language models. Biographical Sketch: Shangqian Gao is a final-year Ph.D. candidate in Electrical and Computer Engineering (ECE) at the Swanson School of Engineering, University of Pittsburgh. Previously, he earned his M.S. degree in Computer System Engineering from Northeastern University in 2017, and his B.S. degree in Electronic Engineering from Xidian University in 2015. He is also a research scientist in Samsung Research America. His current research interests includes efficient and scalable machine learning, cross-modal learning, reinforcement learning and zeroth-order (gradient-free) optimization methods. Shangqian has published more than 20 papers in prestigious conferences and journals such as TPMAI, JMLR, CVPR, ICCV, ECCV, ICLR, NeurIPS, ICML, AAAI, IJCAI, etc. Additionally, he received the President’s Award from Samsung Research America for advancing the efficiency of Large Language Models. Location and Zoom link: 307 Love, or https://fsu.zoom.us/j/91496220517 |
|
