Abstract: In Siamese-network-based self-supervised learning (SSL), multilevel supervision (MLS) is a natural extension to enforce intermediate representations’ consistency against data augmentations.