New Two-Level Ensemble Method and Its Application to Chemical Compounds Properties Prediction

Abstract

The paper considers a new two-level ensemble regression method and its application to prediction problems. At the first stage, the target variable prediction is performed by regression trees included in the lower level optimal ensemble. The aggregated solution is computed by a regression random forest, using the predictions computed by the ensemble trees as input features. The method differs from common ensemble method by a new technique that is used to build trees that are added to ensemble. This technique is based on minimizing a special functional that is the difference of two components. The first component characterizes the approximation of the target variable dependency on input features. The second component is aimed at increasing variance of prognoses calculated by algorithms from ensemble. The developed method implements combination of approaches used in the random forest method and gradient boosting. The paper presents the results of the developed method for predicting the melting points for halides with various compositions, as well as for predicting of the crystal lattices parameters for $$A_{2}BB^{\prime}O_{6}$$ composition compounds.