Transformations based on domain expertise (expert transformations), such as random-resized-crop and color-jitter, have proven critical to the success of contrastive learning techniques such as SimCLR. Recently, several attempts have been made to replace such domain-specific, human-designed transformations with generated views that are learned. However for imagery data, so far none of these view-generation methods has been able to outperform expert transformations. In this work, we tackle a different question: instead of replacing expert transformations with generated views, can we constructively assimilate generated views with expert transformations? We answer this question in the affirmative and propose a view generation method and a simple, effective assimilation method that together improve the state-of-the-art by up to ~3.6% on three different datasets. Importantly, we conduct a detailed empirical study that systematically analyzes a range of view generation and assimilation methods and provides a holistic picture of the efficacy of learned views in contrastive representation learning.
@article{arxiv.2304.00601,
title = {Constructive Assimilation: Boosting Contrastive Learning Performance through View Generation Strategies},
author = {Ligong Han and Seungwook Han and Shivchander Sudalairaj and Charlotte Loh and Rumen Dangovski and Fei Deng and Pulkit Agrawal and Dimitris Metaxas and Leonid Karlinsky and Tsui-Wei Weng and Akash Srivastava},
journal= {arXiv preprint arXiv:2304.00601},
year = {2023}
}
Comments
Accepted at Generative Models for Computer Vision Workshop 2023