HS-KDNet: A Lightweight Network Based on Hierarchical-Split Block and Knowledge Distillation for Fault Diagnosis With Extremely Imbalanced Data

Abstract

Because of the cost, it is unrealistic to sample the failure state for a long time, which makes the data collected from the scenario of engineering usually extremely imbalanced. However, imbalanced training data pose a negative effect on the fault diagnosis algorithms based on the data driven. When the data are extremely imbalanced, this problem becomes more challenging. Furthermore, to reduce the deployment cost, in industrial practice, it is often required that the parameters and computation of the deployed diagnosis model should be within a certain range, which puts forward the requirement of lightweight for diagnosis model. Therefore, in this article, a novel lightweight framework for fault diagnosis with extremely imbalanced data, called HS-KDNet, is proposed. Soft labels generated by knowledge distillation can represent the similarity between categories, i.e., through to learn the soft labels, the information about all categories are considered in each update of the parameters, not only the information about the current samples. Consequently, unlike traditional data re-balancing strategies based on generating pseudo samples, we utilized knowledge distillation to suppress the adverse effects of imbalanced data for the first time. On two classical bearing datasets, the effectiveness and superiority of the proposed HS-KDNet were demonstrated, and the experimental results shown that, except for HS-KDNet, knowledge distillation can significantly inhibit the adverse effects of imbalanced data on other simple models.