Catalytic Role Of Noise And Necessity Of Inductive Biases In The\n Emergence Of Compositional Communication

Abstract

Communication is compositional if complex signals can be represented as a\ncombination of simpler subparts. In this paper, we theoretically show that\ninductive biases on both the training framework and the data are needed to\ndevelop a compositional communication. Moreover, we prove that compositionality\nspontaneously arises in the signaling games, where agents communicate over a\nnoisy channel. We experimentally confirm that a range of noise levels, which\ndepends on the model and the data, indeed promotes compositionality. Finally,\nwe provide a comprehensive study of this dependence and report results in terms\nof recently studied compositionality metrics: topographical similarity,\nconflict count, and context independence.\n