Self-training with Noisy Student improves ImageNet classification (2020)
Contents
- Abstract
- NoisyStudent : Iterative Self-training with Noise
0. Abstract
propose simple self-training method
- step 1) first train an EfficientNet model on labeled ImageNet
- step 2) use it as teacher to generate pseudo labels
- step 3) train a larger EfficientNet as a student model
- with both labeled & unlabeled
1. NoisyStudent : Iterative Self-training with Noise
Step 1) train a teacher model
- with labeled images
Step 2) generate pseudo-labels
- generated with teacher model on unlabeled images
- soft & hard version
Step 3) train a student model
- minimizes the combined cross entropy loss on both labeled images and unlabeled images
Step 4) iterate the process
- putting back the student as a teacher
