3D Point Cloud Classification and Segmentation using 3D Modified Fisher Vector Representation for Convolutional Neural Networks

TLDR

Point clouds are increasingly used to represent 3‑D shapes, yet their irregular structure hinders deep learning, and converting them to voxel grids incurs large memory costs. This work introduces a new 3‑D point‑cloud representation, the 3D Modified Fisher Vector (3DmFV). The 3DmFV combines a discrete grid structure with continuous Fisher‑vector generalization, yielding a compact, efficient representation that enables a novel CNN architecture for classification and part segmentation. Experiments on challenging benchmark datasets show that the proposed method achieves competitive or superior performance compared to state‑of‑the‑art approaches.

Abstract

The point cloud is gaining prominence as a method for representing 3D shapes, but its irregular format poses a challenge for deep learning methods. The common solution of transforming the data into a 3D voxel grid introduces its own challenges, mainly large memory size. In this paper we propose a novel 3D point cloud representation called 3D Modified Fisher Vectors (3DmFV). Our representation is hybrid as it combines the discrete structure of a grid with continuous generalization of Fisher vectors, in a compact and computationally efficient way. Using the grid enables us to design a new CNN architecture for point cloud classification and part segmentation. In a series of experiments we demonstrate competitive performance or even better than state-of-the-art on challenging benchmark datasets.