Generalized Referring expressions can describe one object, several related objects, or none at all. Existing generalized referring segmentation (GRES) models treat all cases alike, predicting a single binary mask and ignoring how linguistic phrases correspond to distinct visual instances. To this end, we reformulate GRES as an instance-level reasoning problem, where the model first predicts multiple instance-aware object queries conditioned on the referring expression, then aligns each with its most relevant phrase. This alignment is enforced by a Phrase-Object Alignment (POA) loss that builds fine-grained correspondence between linguistic phrases and visual instances. Given these aligned object instance queries and their learned relevance scores, the final segmentation and the no-target case are both inferred through a unified relevance-weighted aggregation mechanism. This instance-aware formulation enables explicit phrase-instance grounding, interpretable reasoning, and robust handling of complex or null expressions. Extensive experiments on the gRefCOCO and Ref-ZOM benchmarks demonstrate that our method significantly advances state-of-the-art performance by 3.22% cIoU and 12.25% N-acc.
@article{arxiv.2411.15087,
title = {Phrase-Instance Alignment for Generalized Referring Segmentation},
author = {E-Ro Nguyen and Hieu Le and Dimitris Samaras and Michael S. Ryoo},
journal= {arXiv preprint arXiv:2411.15087},
year = {2026}
}
Comments
Accepted to PVUW - CVPR 2026 Workshop. Webpage: https://eronguyen.github.io/InstAlign/