Preserving face identity is a critical yet persistent challenge in diffusion-based image restoration. While reference faces offer a path forward, existing reference-based methods often fail to fully exploit their potential. This paper introduces a novel approach that maximizes reference face utility for improved face restoration and identity preservation. Our method makes three key contributions: 1) Composite Context, a comprehensive representation that fuses multi-level (high- and low-level) information from the reference face, offering richer guidance than prior singular representations. 2) Hard Example Identity Loss, a novel loss function that leverages the reference face to address the identity learning inefficiencies found in the existing identity loss. 3) A training-free method to adapt the model to multi-reference inputs during inference. The proposed method demonstrably restores high-quality faces and achieves state-of-the-art identity preserving restoration on benchmarks such as FFHQ-Ref and CelebA-Ref-Test, consistently outperforming previous work.
@article{arxiv.2505.21905,
title = {Reference-Guided Identity Preserving Face Restoration},
author = {Mo Zhou and Keren Ye and Viraj Shah and Kangfu Mei and Mauricio Delbracio and Peyman Milanfar and Vishal M. Patel and Hossein Talebi},
journal= {arXiv preprint arXiv:2505.21905},
year = {2025}
}