Maxpolynomial Division with Application To Neural Network Simplification

Abstract

In this work, we further the link between neural networks with piecewise linear activations and tropical algebra. To that end, we introduce the process of Maxpolynomial Division, a geometric method which simulates division of polynomials in the max-plus semiring, while highlighting its key properties and noting its connection to neural networks. Afterwards, we generalize this method and apply it in the context of neural network minimization, for two-layer networks used for binary classification problems, attempting to reduce the size of the hidden layer before the output. A tractable method to find an appropriate divisor and perform the division is introduced and evaluated in the IMDB Movie Review and MNIST datasets, with preliminary experiments demonstrating a capacity of this method to reduce the size of the network, without major loss of performance.