Pointwise Convolutional Neural Networks

TLDR

Deep learning with 3D data such as point clouds and CAD models has attracted significant interest, yet the use of point clouds with convolutional neural networks remains underexplored. The paper proposes a convolutional neural network for semantic segmentation and object recognition of 3D point clouds. The network employs a novel point‑wise convolution operator applied at each point of a point cloud. The fully convolutional design achieves competitive accuracy on semantic segmentation and object recognition tasks.

Abstract

Deep learning with 3D data such as reconstructed point clouds and CAD models has received great research interests recently. However, the capability of using point clouds with convolutional neural network has been so far not fully explored. In this paper, we present a convolutional neural network for semantic segmentation and object recognition with 3D point clouds. At the core of our network is point-wise convolution, a new convolution operator that can be applied at each point of a point cloud. Our fully convolutional network design, while being surprisingly simple to implement, can yield competitive accuracy in both semantic segmentation and object recognition task.