Point Cloud Generation Using Deep Local Features for Augmented and Mixed Reality Contents

Abstract

With the commercialization of 5G network and the mounted of 3D sensor such as ToF on smartphones, augmented and mixed reality (AR/MR) technology has attracted increasing attention. AR/MR contents need 3D data in various models to interact with human users. However, creating a 3D model is a complicated and expensive process involving 3D acquisition, 3D reconstruction, and rendering with computer graphics techniques. To solve that problem, we use an autoencoder to extract local information. We then train the latent space in a generative adversarial network (GAN). The GAN takes local context from the latent variable, and then generates a point cloud of various robust shapes. The proposed method can generate a novel 3D model that can significantly save computational load to render AR/MR contents.