PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture\n Likelihood and Other Modifications

Abstract

PixelCNNs are a recently proposed class of powerful generative models with\ntractable likelihood. Here we discuss our implementation of PixelCNNs which we\nmake available at https://github.com/openai/pixel-cnn. Our implementation\ncontains a number of modifications to the original model that both simplify its\nstructure and improve its performance. 1) We use a discretized logistic mixture\nlikelihood on the pixels, rather than a 256-way softmax, which we find to speed\nup training. 2) We condition on whole pixels, rather than R/G/B sub-pixels,\nsimplifying the model structure. 3) We use downsampling to efficiently capture\nstructure at multiple resolutions. 4) We introduce additional short-cut\nconnections to further speed up optimization. 5) We regularize the model using\ndropout. Finally, we present state-of-the-art log likelihood results on\nCIFAR-10 to demonstrate the usefulness of these modifications.\n