GANS for Sequences of Discrete Elements with the Gumbel-softmax\n Distribution

Abstract

Generative Adversarial Networks (GAN) have limitations when the goal is to\ngenerate sequences of discrete elements. The reason for this is that samples\nfrom a distribution on discrete objects such as the multinomial are not\ndifferentiable with respect to the distribution parameters. This problem can be\navoided by using the Gumbel-softmax distribution, which is a continuous\napproximation to a multinomial distribution parameterized in terms of the\nsoftmax function. In this work, we evaluate the performance of GANs based on\nrecurrent neural networks with Gumbel-softmax output distributions in the task\nof generating sequences of discrete elements.\n