Associate Professor Uchiyama’s co-authored paper has been published in the Elsevier Computers & Graphics Journal. This work is a collaborative research effort between our laboratory, the Federal Rural University of Pernambuco (UFRPE), the Federal University of Pernambuco (UFPE), and Kyushu University.
- João Paulo Lima, Hideaki Uchiyama, Diego Thomas, Veronica Teichrieb,
“Fast direct multi-person radiance fields from sparse input with dense pose priors”
(Abstract) Volumetric radiance fields have been popular in reconstructing small-scale 3D scenes from multi-view images. With additional constraints such as person correspondences, reconstructing a large 3D scene with multiple persons becomes possible. However, existing methods fail for sparse input views or when person correspondences are unavailable. In such cases, the conventional depth image supervision may be insufficient because it only captures the relative position of each person with respect to the camera center. In this paper, we investigate an alternative approach by supervising the optimization framework with a dense pose prior that represents correspondences between the SMPL model and the input images. The core ideas of our approach consist in exploiting dense pose priors estimated from the input images to perform person segmentation and incorporating such priors into the learning of the radiance field. Our proposed dense pose supervision is view-independent, significantly speeding up computational time and improving 3D reconstruction accuracy, with less floaters and noise. We confirm the advantages of our proposed method with extensive evaluation in a subset of the publicly available CMU Panoptic dataset. When training with only five input views, our proposed method achieves an average improvement of 6.1% in PSNR, 3.5% in SSIM, 17.2% in LPIPSvgg, 19.3% in LPIPSalex, and 39.4% in training time.
https://doi.org/10.1016/j.cag.2024.104063
