Point Cloud Generation Using Deep Adversarial Local Features for Augmented and Mixed Reality Contents

Abstract

We present a generative model-based point cloud generation method using deep adversarial local features. The proposed generative adversarial network (GAN) can reduce computational load and increase the accuracy in three-dimensional (3D) acquisition, reconstruction, and rendering processes. To train the proposed GAN, we first optimize the latent space using an autoencoder to extract local features. The training process provides an accurate estimation of local context from the latent variables and robust point cloud generation. The main contribution of this work is a novel deep learning-based 3D point cloud generation, which significantly reduces computational load to render augmented reality (AR) and mixed reality (MR) contents. Additional contribution in the deep learning field is twofold: i) The autoencoder in the proposed network avoids the vanishing gradient problem using hierarchically linked features in different layers, and ii) the complexity of the network is significantly reduced by removing the transformation network that estimates the affine transformation matrix of the point cloud.