English
Related papers

Related papers: GLDiTalker: Speech-Driven 3D Facial Animation with…

200 papers

Realistic, high-fidelity 3D facial animations are crucial for expressive avatar systems in human-computer interaction and accessibility. Although prior methods show promising quality, their reliance on the mesh domain limits their ability…

Computer Vision and Pattern Recognition · Computer Science 2025-07-22 Alexandre Symeonidis-Herzig , Özge Mercanoğlu Sincan , Richard Bowden

Speech-driven 3D facial animation is challenging due to the diversity in speaking styles and the limited availability of 3D audio-visual data. Speech predominantly dictates the coarse motion trends of the lip region, while specific styles…

Multimedia · Computer Science 2025-03-14 An Yang , Chenyu Liu , Pengcheng Xia , Jun Du

The generation of stylistic 3D facial animations driven by speech presents a significant challenge as it requires learning a many-to-many mapping between speech, style, and the corresponding natural facial motion. However, existing methods…

Computer Vision and Pattern Recognition · Computer Science 2024-05-15 Zhiyao Sun , Tian Lv , Sheng Ye , Matthieu Lin , Jenny Sheng , Yu-Hui Wen , Minjing Yu , Yong-Jin Liu

Speech-driven 3D facial animation has been widely explored, with applications in gaming, character animation, virtual reality, and telepresence systems. State-of-the-art methods deform the face topology of the target actor to sync the input…

Computer Vision and Pattern Recognition · Computer Science 2023-01-03 Balamurugan Thambiraja , Ikhsanul Habibie , Sadegh Aliakbarian , Darren Cosker , Christian Theobalt , Justus Thies

Joint audio-video generation models have shown that unified generation yields stronger cross-modal coherence than cascaded approaches. However, existing models couple modalities throughout denoising via pervasive attention, treating…

Computer Vision and Pattern Recognition · Computer Science 2026-04-28 Zhen Ye , Xu Tan , Aoxiong Yin , Hongzhan Lin , Guangyan Zhang , Peiwen Sun , Yiming Li , Chi-Min Chan , Wei Ye , Shikun Zhang , Wei Xue

Speech-driven 3D facial animation aims to synthesize vivid facial animations that accurately synchronize with speech and match the unique speaking style. However, existing works primarily focus on achieving precise lip synchronization while…

Computer Vision and Pattern Recognition · Computer Science 2023-12-19 Hui Fu , Zeqing Wang , Ke Gong , Keze Wang , Tianshui Chen , Haojie Li , Haifeng Zeng , Wenxiong Kang

Audio-driven talking head generation is a core component of digital avatars, and 3D Gaussian Splatting has shown strong performance in real-time rendering of high-fidelity talking heads. However, achieving precise control over fine-grained…

Computer Vision and Pattern Recognition · Computer Science 2026-02-11 Shaoyang Xie , Xiaofeng Cong , Baosheng Yu , Zhipeng Gui , Jie Gui , Yuan Yan Tang , James Tin-Yau Kwok

Creating high-quality, generalizable speech-driven 3D talking heads remains a persistent challenge. Previous methods achieve satisfactory results for fixed viewpoints and small-scale audio variations, but they struggle with large head…

Computer Vision and Pattern Recognition · Computer Science 2025-07-11 Wentao Hu , Shunkai Li , Ziqiao Peng , Haoxian Zhang , Fan Shi , Xiaoqiang Liu , Pengfei Wan , Di Zhang , Hui Tian

Audio-driven 3D facial animation aims to map input audio to realistic facial motion. Despite significant progress, limitations arise from inconsistent 3D annotations, restricting previous models to training on specific annotations and…

Computer Vision and Pattern Recognition · Computer Science 2024-08-02 Xiangyu Fan , Jiaqi Li , Zhiqian Lin , Weiye Xiao , Lei Yang

Speech-driven 3D talking heads generation has emerged as a significant area of interest among researchers, presenting numerous challenges. Existing methods are constrained by animating faces with fixed topologies, wherein point-wise…

Computer Vision and Pattern Recognition · Computer Science 2024-09-26 Federico Nocentini , Thomas Besnier , Claudio Ferrari , Sylvain Arguillere , Stefano Berretti , Mohamed Daoudi

Talking face generation aims to synthesize realistic speaking portraits from a single image, yet existing methods often rely on explicit optical flow and local warping, which fail to model complex global motions and cause identity drift. We…

Computer Vision and Pattern Recognition · Computer Science 2025-12-01 Bo Chen , Tao Liu , Qi Chen , Xie Chen , Zilong Zheng

In recent years, audio-driven 3D facial animation has gained significant attention, particularly in applications such as virtual reality, gaming, and video conferencing. However, accurately modeling the intricate and subtle dynamics of…

Computer Vision and Pattern Recognition · Computer Science 2023-11-14 Guinan Su , Yanwu Yang , Zhifeng Li

Audio-driven talking head generation has drawn much attention in recent years, and many efforts have been made in lip-sync, expressive facial expressions, natural head pose generation, and high video quality. However, no model has yet led…

Computer Vision and Pattern Recognition · Computer Science 2023-12-08 Xusen Sun , Longhao Zhang , Hao Zhu , Peng Zhang , Bang Zhang , Xinya Ji , Kangneng Zhou , Daiheng Gao , Liefeng Bo , Xun Cao

High-quality, real-time talking head synthesis remains a fundamental challenge in computer vision. Existing reconstruction- and rendering-based methods typically rely on identity-specific models, limiting cross-identity generalization. To…

Computer Vision and Pattern Recognition · Computer Science 2026-05-12 Peng Jia , Zhen Xiao , Jia Li , Xueliang Liu , Zhenzhen Hu , Lingyun Yu

Speech-driven 3D facial animation has been widely studied, yet there is still a gap to achieving realism and vividness due to the highly ill-posed nature and scarcity of audio-visual data. Existing works typically formulate the cross-modal…

Computer Vision and Pattern Recognition · Computer Science 2023-04-04 Jinbo Xing , Menghan Xia , Yuechen Zhang , Xiaodong Cun , Jue Wang , Tien-Tsin Wong

Audio-driven emotional 3D facial animation aims to generate synchronized lip movements and vivid facial expressions. However, most existing approaches focus on static and predefined emotion labels, limiting their diversity and naturalness.…

Computer Vision and Pattern Recognition · Computer Science 2025-08-15 Chang Liu , Ye Pan , Chenyang Ding , Susanto Rahardja , Xiaokang Yang

Recent studies in speech-driven 3D talking head generation have achieved convincing results in verbal articulations. However, generating accurate lip-syncs degrades when applied to input speech in other languages, possibly due to the lack…

Computer Vision and Pattern Recognition · Computer Science 2024-06-21 Kim Sung-Bin , Lee Chae-Yeon , Gihun Son , Oh Hyun-Bin , Janghoon Ju , Suekyeong Nam , Tae-Hyun Oh

Although existing speech-driven talking face generation methods achieve significant progress, they are far from real-world application due to the avatar-specific training demand and unstable lip movements. To address the above issues, we…

Computer Vision and Pattern Recognition · Computer Science 2023-12-13 Haiming Zhang , Zhihao Yuan , Chaoda Zheng , Xu Yan , Baoyuan Wang , Guanbin Li , Song Wu , Shuguang Cui , Zhen Li

Recent advancements in video diffusion models have significantly enhanced audio-driven portrait animation. However, current methods still suffer from flickering, identity drift, and poor audio-visual synchronization. These issues primarily…

Computer Vision and Pattern Recognition · Computer Science 2025-12-19 Zhenjie Liu , Jianzhang Lu , Renjie Lu , Cong Liang , Shangfei Wang

Recent diffusion-based talking face generation models have demonstrated impressive potential in synthesizing videos that accurately match a speech audio clip with a given reference identity. However, existing approaches still encounter…

Computer Vision and Pattern Recognition · Computer Science 2025-10-16 Xingpei Ma , Jiaran Cai , Yuansheng Guan , Shenneng Huang , Qiang Zhang , Shunsi Zhang