In this paper, we introduce CalliffusionV2, a novel system designed to produce natural Chinese calligraphy with flexible multi-modal control. Unlike previous approaches that rely solely on image or text inputs and lack fine-grained control, our system leverages both images to guide generations at fine-grained levels and natural language texts to describe the features of generations. CalliffusionV2 excels at creating a broad range of characters and can quickly learn new styles through a few-shot learning approach. It is also capable of generating non-Chinese characters without prior training. Comprehensive tests confirm that our system produces calligraphy that is both stylistically accurate and recognizable by neural network classifiers and human evaluators.
@article{arxiv.2410.03787,
title = {CalliffusionV2: Personalized Natural Calligraphy Generation with Flexible Multi-modal Control},
author = {Qisheng Liao and Liang Li and Yulang Fei and Gus Xia},
journal= {arXiv preprint arXiv:2410.03787},
year = {2024}
}