2021 IEEE International Conference on Acoustics, Speech and Signal Processing

6-11 June 2021 • Toronto, Ontario, Canada

Extracting Knowledge from Information

2021 IEEE International Conference on Acoustics, Speech and Signal Processing

6-11 June 2021 • Toronto, Ontario, Canada

Extracting Knowledge from Information
Login Paper Search My Schedule Paper Index Help

My ICASSP 2021 Schedule

Note: Your custom schedule will not be saved unless you create a new account or login to an existing account.
  1. Create a login based on your email (takes less than one minute)
  2. Perform 'Paper Search'
  3. Select papers that you desire to save in your personalized schedule
  4. Click on 'My Schedule' to see the current list of selected papers
  5. Click on 'Printable Version' to create a separate window suitable for printing (the header and menu will appear, but will not actually print)

Paper Detail

Paper IDMMSP-2.3
Paper Title Teacher-Assisted Mini-Batch Sampling for Blind Distillation using Metric Learning
Authors Nakamasa Inoue, Tokyo Institute of Technology, Japan
SessionMMSP-2: Deep Learning for Multimedia Analysis and Processing
LocationGather.Town
Session Time:Tuesday, 08 June, 14:00 - 14:45
Presentation Time:Tuesday, 08 June, 14:00 - 14:45
Presentation Poster
Topic Multimedia Signal Processing: Emerging Areas in Multimedia
IEEE Xplore Open Preview  Click here to view in IEEE Xplore
Abstract This paper addresses the problem of blind distillation, which aims to train a student model with unlabeled data under the supervision of a pre-trained teacher model. The proposed framework introduces metric learning to blind distillation. Specifically, teacher-assisted mini-batch (TAM) sampling is proposed, which makes triplets of anchor, positive and negative samples on unlabeled data by using the teacher's knowledge. In addition, we propose a metric-based loss, namely Contrastive Additive Margin (CAM) Softmax loss, which efficiently uses all combinations of triplets on each mini-batch obtained by TAM sampling. In experiments, we show the effectiveness of the proposed framework on face and speaker verification tasks, where student models are trained on unlabeled VoxCeleb videos with a teacher model pre-trained on VGGFace2 images.