2021 IEEE International Conference on Acoustics, Speech and Signal Processing

6-11 June 2021 • Toronto, Ontario, Canada

Extracting Knowledge from Information

2021 IEEE International Conference on Acoustics, Speech and Signal Processing

6-11 June 2021 • Toronto, Ontario, Canada

Extracting Knowledge from Information
Login Paper Search My Schedule Paper Index Help

My ICASSP 2021 Schedule

Note: Your custom schedule will not be saved unless you create a new account or login to an existing account.
  1. Create a login based on your email (takes less than one minute)
  2. Perform 'Paper Search'
  3. Select papers that you desire to save in your personalized schedule
  4. Click on 'My Schedule' to see the current list of selected papers
  5. Click on 'Printable Version' to create a separate window suitable for printing (the header and menu will appear, but will not actually print)

Paper Detail

Paper IDASPS-7.5
Paper Title SQWA: STOCHASTIC QUANTIZED WEIGHT AVERAGING FOR IMPROVING THE GENERALIZATION CAPABILITY OF LOW-PRECISION DEEP NEURAL NETWORKS
Authors Sungho Shin, Yoonho Boo, Wonyong Sung, Seoul National University, South Korea
SessionASPS-7: Data Science & Machine Learning
LocationGather.Town
Session Time:Thursday, 10 June, 16:30 - 17:15
Presentation Time:Thursday, 10 June, 16:30 - 17:15
Presentation Poster
Topic Applied Signal Processing Systems: Signal Processing Systems [DIS-EMSA]
IEEE Xplore Open Preview  Click here to view in IEEE Xplore
Abstract Low-precision deep neural networks (DNNs) are very needed for efficient implementations, but severe quantization of weights often sacrifices the generalization capability and lowers the test accuracy. We present a new quantized neural network optimization approach, stochastic quantized weight averaging (SQWA), to design low-precision DNNs with good generalization capability using model averaging. The proposed approach includes (1) floating-point model training, (2) direct quantization of weights, (3) capturing multiple low precision models during retraining with cyclical learning rates, (4) averaging the captured models, and (5) re-quantizing the averaged model and fine-tuning it with low-learning rates. With SQWA training, we could develop the best performing QDNNs for image classification on ImageNet datasets and also for semantic segmentation on Pascal VOC 2012 dataset.