Learning Theory of Distributed Regression with Bias Corrected\n Regularization Kernel Network

Zheng-Chu Guo, Lei Shi, Qiang Wu

arXiv (Cornell University) · 2017 · 35 citations · 0 references

DOIFull text

Open access

Concepts

Abstract

Distributed learning is an effective way to analyze big data. In distributed\nregression, a typical approach is to divide the big data into multiple blocks,\napply a base regression algorithm on each of them, and then simply average the\noutput functions learnt from these blocks. Since the average process will\ndecrease the variance, not the bias, bias correction is expected to improve the\nlearning performance if the base regression algorithm is a biased one.\nRegularization kernel network is an effective and widely used method for\nnonlinear regression analysis. In this paper we will investigate a bias\ncorrected version of regularization kernel network. We derive the error bounds\nwhen it is applied to a single data set and when it is applied as a base\nalgorithm in distributed regression. We show that, under certain appropriate\nconditions, the optimal learning rates can be reached in both situations.\n