Gradient boosting machines, a tutorial

TLDR

Gradient boosting machines are powerful machine‑learning techniques with wide practical success and high customizability to application needs, including different loss functions. This article gives a tutorial introduction into the methodology of gradient boosting methods. The tutorial combines theoretical discussion with descriptive examples and illustrations covering all stages of gradient boosting model design, and presents practical applications that are comprehensively analyzed. The article discusses how to handle model complexity and presents practical examples of gradient boosting applications that are comprehensively analyzed.

Abstract

Gradient boosting machines are a family of powerful machine-learning techniques that have shown considerable success in a wide range of practical applications. They are highly customizable to the particular needs of the application, like being learned with respect to different loss functions. This article gives a tutorial introduction into the methodology of gradient boosting methods. A theoretical information is complemented with many descriptive examples and illustrations which cover all the stages of the gradient boosting model design. Considerations on handling the model complexity are discussed. A set of practical examples of gradient boosting applications are presented and comprehensively analyzed.