Dataset Condensation (DC) seeks to select or distill samples from large datasets into smaller subsets while preserving performance on target tasks. Existing methods primarily focus on pruning or synthesizing data in the same format as the original dataset, typically being the input data and corresponding labels. However, in DC settings, we find it is possible to synthesize more information beyond the data-label pair as an additional learning target to facilitate model training. In this paper, we introduce Dataset Condensation using Privileged Information (DCPI), which enriches DC by synthesizing privileged information alongside the reduced dataset. This privileged information can take the form of feature labels or attention labels, providing auxiliary supervision to improve model learning. Our findings reveal that effective feature labels must balance between being overly discriminative and excessively diverse, with a moderate level proves optimal for improving the reduced dataset's efficacy. Extensive experiments on ImageNet-1K, CIFAR-10/100 and Tiny ImageNet demonstrate that DCPI integrates seamlessly with existing dataset condensation methods, offering significant performance gains.
@article{arxiv.2410.01611,
title = {DRUPI: Dataset Reduction Using Privileged Information},
author = {Shaobo Wang and Youxin Jiang and Tianle Niu and Yantai Yang and Ruiji Zhang and Shuhao Hu and Shuaiyu Zhang and Chenghao Sun and Weiya Li and Conghui He and Xuming Hu and Linfeng Zhang},
journal= {arXiv preprint arXiv:2410.01611},
year = {2026}
}