Deep feature synthesis: Towards automating data science endeavors

TLDR

The paper develops the Data Science Machine to automatically generate predictive models from raw data. It introduces Deep Feature Synthesis, which traverses relational data to create features via sequential mathematical functions, and couples this with a general machine‑learning pipeline tuned by a Gaussian Copula process. In three Kaggle competitions, the system outperformed 615 of 906 teams, winning the majority in two contests and achieving up to 95.7% of the top score.

Abstract

In this paper, we develop the Data Science Machine, which is able to derive predictive models from raw data automatically. To achieve this automation, we first propose and develop the Deep Feature Synthesis algorithm for automatically generating features for relational datasets. The algorithm follows relationships in the data to a base field, and then sequentially applies mathematical functions along that path to create the final feature. Second, we implement a generalizable machine learning pipeline and tune it using a novel Gaussian Copula process based approach. We entered the Data Science Machine in 3 data science competitions that featured 906 other data science teams. Our approach beats 615 teams in these data science competitions. In 2 of the 3 competitions we beat a majority of competitors, and in the third, we achieved 94% of the best competitor's score. In the best case, with an ongoing competition, we beat 85.6% of the teams and achieved 95.7% of the top submissions score.