Ensemble Deep Random Vector Functional Link Neural Network Based on Fuzzy Inference System

Abstract

The ensemble deep random vector functional link (edRVFL) neural network has demonstrated the ability to address the limitations of conventional artificial neural networks. However, since edRVFL generates features for its hidden layers through random projection, it can potentially lose intricate features or fail to capture certain nonlinear features in its base models (hidden layers). To enhance the feature learning capabilities of edRVFL, we propose a novel edRVFL based on fuzzy inference system (edRVFL-FIS). The proposed edRVFL-FIS leverages the capabilities of two emerging domains, namely deep learning and ensemble approaches, with the intrinsic IF–THEN properties of FIS and produces rich feature representation to train the ensemble model. Each base model of the proposed edRVFL-FIS encompasses the following two key feature augmentation components: 1) unsupervised fuzzy layer features and 2) supervised defuzzified features. The edRVFL-FIS model incorporates diverse clustering methods (R-means, K-means, fuzzy C-means) to establish fuzzy layer rules, resulting in the following three model variations: 1) edRVFL-FIS-R, 2) edRVFL-FIS-K, and 3) edRVFL-FIS-C with distinct fuzzified features and defuzzified features. Within the framework of edRVFL-FIS, each base model utilizes the original, hidden layer, and defuzzified features to make predictions. Experimental results, statistical tests, discussions, and analyses conducted across UCI and NDC datasets consistently demonstrate the superior performance of all variations of the proposed edRVFL-FIS model over baseline models such as fuzzy broad learning system.