This work considers merging two independent models, TTS and A2F, into a unified model to enable internal feature transfer, thereby improving the consistency between audio and facial expressions generated from text. We also discuss the extension of the emotion control mechanism from TTS to the joint model. This work does not aim to showcase generation quality; instead, from a system design perspective, it validates the feasibility of reusing intermediate representations from TTS for joint modeling of speech and facial expressions, and provides engineering practice references for subsequent speech expression co-design. The project code has been open source at: https://github.com/GoldenFishes/UniTAF
@article{arxiv.2602.15651,
title = {UniTAF: A Modular Framework for Joint Text-to-Speech and Audio-to-Face Modeling},
author = {Qiangong Zhou and Nagasaka Tomohiro},
journal= {arXiv preprint arXiv:2602.15651},
year = {2026}
}
Comments
We have identified inaccuracies in some results that require further verification. To avoid misleading the research community, we are temporarily withdrawing the paper