About boosted learning of nonparametric estimators

Versatility of identification methods allows to apply them in different technical areas (including the aerospace industry), as well as in medicine, economics, etc. In recent years, ensemble learning is becoming one of the most common methods of identification. Ensemble methods train multiple learners and further combine their use. One of the main tasks of combining multiple models with the same type is to eliminate certain drawbacks of individual models. This paper deals with some peculiar properties of the Nadaraya-watson kernel estimator. These peculiar properties are related with existence of sparse areas in the space of input variables (some regions contain a small number of observations in the training set) and with the behavior of the Nadaraya-watson kernel estimator near the boundary of the input variables space. The ensemble learning approach proposed by the authors is based on boosted learning of nonparametric estimators. There is a formalized approach to ensemble building with configurable parameters and there are some guidelines for choosing these parameters. The numerical researches shows that proposed boosted ensemble is significantly more accurate than a single Nadaraya-watson kernel estimator both in case of sparse areas in the space of input variables, and in case of areas with a large number of observations in the training set. Also the numerical research demonstrates high accuracy of proposed boosted ensemble near the boundary of the input variables space and shows the possibility of using boosted ensemble in the extrapolation problem.