Generating Part-Aware Editable 3D Shapes without 3D Supervision

Abstract

Impressive progress in generative models and implicit representations gave rise to methods that can generate 3D shapes of high quality. However, being able to locally con-trol and edit shapes is another essential property that can unlock several content creation applications. Local control can be achieved with part-aware models, but existing meth-ods require 3D supervision and cannot produce textures. In this work, we devise PartNeRF, a novel part-aware gener-ative model for editable 3D shape synthesis that does not require any explicit 3D supervision. Our model generates objects as a set of locally defined NeRFs, augmented with an affine transformation. This enables several editing op-erations such as applying transformations on parts, mixing parts from different objects etc. To ensure distinct, manip-ulable parts we enforce a hard assignment of rays to parts that makes sure that the color of each ray is only determined by a single NeRF. As a result, altering one part does not af-fect the appearance of the others. Evaluations on various ShapeNet categories demonstrate the ability of our model to generate editable 3D objects of improved fidelity, compared to previous part-based generative approaches that require 3D supervision or models relying on NeRFs.