On Removing Supervision from Contrastive Self-Supervised Learning