3D keypoint detection by light field scale-depth space analysis

Abstract

We present a method for 3D keypoint detection from light field data, typically obtained by planar camera arrays or plenoptic cameras. The proposed approach is based on construction of novel light field scale-depth spaces that are designed to leverage the specific properties of light fields. The constructed scale-depth spaces are based on a modified Gaussian kernel that is parametrized both in terms of scale of objects recorded by the light field and in terms of objects' depth. We prove theoretically that the new scale-depth space formulation and its spatial derivative satisfy the scale invariance property for all depths. By finding local extrema in such scale-depth spaces we locate 3D keypoints (such as 3D edges) and show that they outperform SURF keypoints on a 3D structure estimation task, both in accuracy and computational efficiency.