English
Related papers

Related papers: Training and Tuning Generative Neural Radiance Fie…

200 papers

We propose GazeNeRF, a 3D-aware method for the task of gaze redirection. Existing gaze redirection methods operate on 2D images and struggle to generate 3D consistent results. Instead, we build on the intuition that the face region and…

Computer Vision and Pattern Recognition · Computer Science 2023-03-30 Alessandro Ruzzi , Xiangwei Shi , Xi Wang , Gengyan Li , Shalini De Mello , Hyung Jin Chang , Xucong Zhang , Otmar Hilliges

Recently, Neural Radiance Fields (NeRF) have emerged as a potent method for synthesizing novel views from a dense set of images. Despite its impressive performance, NeRF is plagued by its necessity for numerous calibrated views and its…

Computer Vision and Pattern Recognition · Computer Science 2023-03-13 Jiayang Bai , Letian Huang , Wen Gong , Jie Guo , Yanwen Guo

Considering the problem of novel view synthesis (NVS) from only a set of 2D images, we simplify the training process of Neural Radiance Field (NeRF) on forward-facing scenes by removing the requirement of known or pre-computed camera…

Computer Vision and Pattern Recognition · Computer Science 2022-04-07 Zirui Wang , Shangzhe Wu , Weidi Xie , Min Chen , Victor Adrian Prisacariu

Recent works have shown that 3D-aware GANs trained on unstructured single image collections can generate multiview images of novel instances. The key underpinnings to achieve this are a 3D radiance field generator and a volume rendering…

Computer Vision and Pattern Recognition · Computer Science 2023-10-12 Jianfeng Xiang , Jiaolong Yang , Yu Deng , Xin Tong

Audio-driven talking head generation is advancing from 2D to 3D content. Notably, Neural Radiance Field (NeRF) is in the spotlight as a means to synthesize high-quality 3D talking head outputs. Unfortunately, this NeRF-based approach…

Computer Vision and Pattern Recognition · Computer Science 2024-05-13 Gihoon Kim , Kwanggyoon Seo , Sihun Cha , Junyong Noh

We propose NeRF-Insert, a NeRF editing framework that allows users to make high-quality local edits with a flexible level of control. Unlike previous work that relied on image-to-image models, we cast scene editing as an in-painting…

Computer Vision and Pattern Recognition · Computer Science 2024-05-01 Benet Oriol Sabat , Alessandro Achille , Matthew Trager , Stefano Soatto

As recent advances in Neural Radiance Fields (NeRF) have enabled high-fidelity 3D face reconstruction and novel view synthesis, its manipulation also became an essential task in 3D vision. However, existing manipulation methods require…

Computer Vision and Pattern Recognition · Computer Science 2023-08-21 Sungwon Hwang , Junha Hyung , Daejin Kim , Min-Jung Kim , Jaegul Choo

Stylized view generation of scenes captured casually using a camera has received much attention recently. The geometry and appearance of the scene are typically captured as neural point sets or neural radiance fields in the previous work.…

Computer Vision and Pattern Recognition · Computer Science 2022-12-20 Rahul Goel , Sirikonda Dhawal , Saurabh Saini , P. J. Narayanan

Animating high-fidelity video portrait with speech audio is crucial for virtual reality and digital entertainment. While most previous studies rely on accurate explicit structural information, recent works explore the implicit scene…

Computer Vision and Pattern Recognition · Computer Science 2022-02-11 Xian Liu , Yinghao Xu , Qianyi Wu , Hang Zhou , Wayne Wu , Bolei Zhou

Neural Radiance Fields (NeRFs) are a powerful representation for modeling a 3D scene as a continuous function. Though NeRF is able to render complex 3D scenes with view-dependent effects, few efforts have been devoted to exploring its…

Computer Vision and Pattern Recognition · Computer Science 2022-11-18 Yifan Jiang , Peter Hedman , Ben Mildenhall , Dejia Xu , Jonathan T. Barron , Zhangyang Wang , Tianfan Xue

Novel view synthesis is an essential functionality for enabling immersive experiences in various Augmented- and Virtual-Reality (AR/VR) applications, for which generalizable Neural Radiance Fields (NeRFs) have gained increasing popularity…

Computer Vision and Pattern Recognition · Computer Science 2025-01-07 Yonggan Fu , Zhifan Ye , Jiayi Yuan , Shunyao Zhang , Sixu Li , Haoran You , Yingyan Celine Lin

Nerf-based Generative models have shown impressive capacity in generating high-quality images with consistent 3D geometry. Despite successful synthesis of fake identity images randomly sampled from latent space, adopting these models for…

Computer Vision and Pattern Recognition · Computer Science 2022-12-01 Yu Yin , Kamran Ghasedi , HsiangTao Wu , Jiaolong Yang , Xin Tong , Yun Fu

Neural Radiance Fields (NeRFs) have proven to be powerful 3D representations, capable of high quality novel view synthesis of complex scenes. While NeRFs have been applied to graphics, vision, and robotics, problems with slow rendering…

Computer Vision and Pattern Recognition · Computer Science 2023-10-30 Tristan Aumentado-Armstrong , Ashkan Mirzaei , Marcus A. Brubaker , Jonathan Kelly , Alex Levinshtein , Konstantinos G. Derpanis , Igor Gilitschenski

Gaze-annotated facial data is crucial for training deep neural networks (DNNs) for gaze estimation. However, obtaining these data is labor-intensive and requires specialized equipment due to the challenge of accurately annotating the gaze…

Computer Vision and Pattern Recognition · Computer Science 2024-06-03 Nerea Aranjuelo , Siyu Huang , Ignacio Arganda-Carreras , Luis Unzueta , Oihana Otaegui , Hanspeter Pfister , Donglai Wei

Data-driven generative 3D face models are used to compactly encode facial shape data into meaningful parametric representations. A desirable property of these models is their ability to effectively decouple natural sources of variation, in…

Computer Vision and Pattern Recognition · Computer Science 2019-09-10 Victoria Fernandez Abrevaya , Adnane Boukhayma , Stefanie Wuhrer , Edmond Boyer

Neural radiance fields (NeRF) achieve highly photo-realistic novel-view synthesis, but it's a challenging problem to edit the scenes modeled by NeRF-based methods, especially for dynamic scenes. We propose editable neural radiance fields…

Computer Vision and Pattern Recognition · Computer Science 2023-03-29 Chengwei Zheng , Wenbin Lin , Feng Xu

Generative models can be used to synthesize 3D objects of high quality and diversity. However, there is typically no control over the properties of the generated object.This paper proposes a novel generative adversarial network (GAN) setup…

Computer Vision and Pattern Recognition · Computer Science 2022-09-12 Larissa T. Triess , Andre Bühler , David Peter , Fabian B. Flohr , J. Marius Zöllner

Recent improvements in Generative Adversarial Neural Networks (GANs) have shown their ability to generate higher quality samples as well as to learn good representations for transfer learning. Most of the representation learning methods…

Audio and Speech Processing · Electrical Eng. & Systems 2020-06-02 Kazi Nazmul Haque , Rajib Rana , John H. L. Hansen , Björn Schuller

Facial recognition using deep convolutional neural networks relies on the availability of large datasets of face images. Many examples of identities are needed, and for each identity, a large variety of images are needed in order for the…

Computer Vision and Pattern Recognition · Computer Science 2021-04-01 Richard T. Marriott , Sami Romdhani , Liming Chen

Recent advances in Neural Radiance Fields (NeRFs) have made it possible to reconstruct and reanimate dynamic portrait scenes with control over head-pose, facial expressions and viewing direction. However, training such models assumes…

Computer Vision and Pattern Recognition · Computer Science 2023-09-22 ShahRukh Athar , Zhixin Shu , Zexiang Xu , Fujun Luan , Sai Bi , Kalyan Sunkavalli , Dimitris Samaras