StyleGANs are at the forefront of controllable image generation as they produce a latent space that is semantically disentangled, making it suitable for image editing and manipulation. However, the performance of StyleGANs severely degrades when trained via class-conditioning on large-scale long-tailed datasets. We find that one reason for degradation is the collapse of latents for each class in the W latent space. With NoisyTwins, we first introduce an effective and inexpensive augmentation strategy for class embeddings, which then decorrelates the latents based on self-supervision in the W space. This decorrelation mitigates collapse, ensuring that our method preserves intra-class diversity with class-consistency in image generation. We show the effectiveness of our approach on large-scale real-world long-tailed datasets of ImageNet-LT and iNaturalist 2019, where our method outperforms other methods by ∼19% on FID, establishing a new state-of-the-art.
@article{arxiv.2304.05866,
title = {NoisyTwins: Class-Consistent and Diverse Image Generation through StyleGANs},
author = {Harsh Rangwani and Lavish Bansal and Kartik Sharma and Tejan Karmali and Varun Jampani and R. Venkatesh Babu},
journal= {arXiv preprint arXiv:2304.05866},
year = {2023}
}