Most multimodal models treat every negative pair alike, ignoring the ambiguous negatives that differ from the positive by only a small detail. We propose Boundary-Aware Curriculum with Local Attention (BACL), a lightweight add-on that turns these borderline cases into a curriculum signal. A Boundary-aware Negative Sampler gradually raises difficulty, while a Contrastive Local Attention loss highlights where the mismatch occurs. The two modules are fully differentiable and work with any off-the-shelf dual encoder. Theory predicts a fast O(1/n) error rate; practice shows up to +32% R@1 over CLIP and new SOTA on four large-scale benchmarks, all without extra labels.
@article{arxiv.2511.08399,
title = {Aligning by Misaligning: Boundary-aware Curriculum Learning for Multimodal Alignment},
author = {Hua Ye and Hang Ding and Siyuan Chen and Yiyang Jiang and Changyuan Zhang and Xuan Zhang},
journal= {arXiv preprint arXiv:2511.08399},
year = {2026}
}
Comments
24 pages, 6 figures, 5 tables. Submitted to NeurIPS 2025