DFLOWFS: Deep Fast-Learning Optimal Weight Fuzzy System with bottom-up hierarchical structure for high-dimensional data regression

Abstract

As a model for reasoning and decision-making based on fuzzy rules, fuzzy systems have high interpretability. However, when the data dimension increases, the fuzzy system will face the problem of “rule explosion”, making it difficult to learn and predict effectively. In this paper, the fuzzy system trained by the FLOWFS (Fast-Learning with Optimal Weights for Fuzzy Systems) algorithm is used as sub-module in the deep fuzzy system, and the deep fuzzy system DFLOWFS (Deep FLOWFS) is constructed from the bottom-up hierarchical structure as the following three steps. 1) The FLOWFS algorithm assigns weight attributions to each fuzzy rule, and the rule weights are trained by the least square method with regularization terms to shorten training time and improve accuracy. 2) Three strategies of dividing high-dimensional inputs into multiple low-dimensional inputs are proposed as sequential division, random division and correlation division. Then, it is verified by experiments that the correlation division has the best performance. 3) The sub-module discarding method is proposed to discard the sub-modules with poor performance to have a maximum improvement of 13.8% compared to the DFLOWFS without using the sub-module discarding method. Then, the optimized DFLOWFS is verified and compared with the other three classic regression models on the three UCI datasets. Experiments show that with the increase of the data dimension, DFLOWFS not only have good interpretability but also have good accuracy. Furthermore, DFLOWFS performs best among all models in comprehensive scores, with good learning ability and generalization ability. Therefore, the proposed strategies with hierarchical structure for optimal shallow fuzzy systems are effective, which give a new insight for fuzzy system research.