What Shape Are Dolphins? Building 3D Morphable Models from 2D Images

TLDR

3D morphable models provide a low‑dimensional parameterization of 3D object classes, but are usually built from 3D scans, making them impractical for many animal classes. The authors construct a 3D morphable model from 2D images by fitting a linear combination of subdivision surfaces to silhouettes and key points, starting from a rough rigid mean shape and using a novel combined continuous‑discrete optimization. The approach yields high‑quality models for several natural object classes from limited 2D data, even without surface texture, demonstrating that only minimal user interaction is needed.

Abstract

3D morphable models are low-dimensional parameterizations of 3D object classes which provide a powerful means of associating 3D geometry to 2D images. However, morphable models are currently generated from 3D scans, so for general object classes such as animals they are economically and practically infeasible. We show that, given a small amount of user interaction (little more than that required to build a conventional morphable model), there is enough information in a collection of 2D pictures of certain object classes to generate a full 3D morphable model, even in the absence of surface texture. The key restriction is that the object class should not be strongly articulated, and that a very rough rigid model should be provided as an initial estimate of the “mean shape.” The model representation is a linear combination of subdivision surfaces, which we fit to image silhouettes and any identifiable key points using a novel combined continuous-discrete optimization strategy. Results are demonstrated on several natural object classes, and show that models of rather high quality can be obtained from this limited information.