Variational Positive-Incentive Noise: How Noise Benefits Models

Abstract

A large number of works aim to alleviate the impact of noise due to an underlying conventional assumption of the negative role of noise. However, some existing works show that the assumption does not always hold. In this paper, we investigate how to benefit the classical models by random noise under the framework of Positive-incentive Noise (Pi-Noise) (Li, 2024). Since the ideal objective of Pi-Noise is intractable, we propose to optimize its variational bound instead, namely variational Pi-Noise (VPN). With the variational inference, a VPN generator implemented by neural networks is designed for enhancing base models and simplifying the inference of base models, without changing the architecture of base models. Benefiting from the independent design of base models and VPN generators, the VPN generator can work with most existing models. From the extensive experiments on different base models (including linear models, ResNet, ViT, etc.) it is shown that the proposed VPN generator can improve the base models. It is appealing that the trained VPN generator prefers to blur the irrelevant ingredients in complicated images, which meets our expectations.