On the Stepwise Nature of Self-Supervised Learning – The Berkeley Artificial Intelligence Research Blog
6 Mins read
Figure 1: stepwise behavior in self-supervised learning. When training common SSL algorithms, we find that the loss descends in a stepwise fashion…