ModelTracker

TLDR

Model building in machine learning is an iterative process, yet performance analysis and debugging often require a disruptive cognitive switch that discourages an informed approach. We present ModelTracker, an interactive visualization that consolidates numerous traditional summary statistics and graphs while displaying example‑level performance and enabling direct error examination and debugging. ModelTracker integrates these statistics and graphs into a single interactive interface, allowing users to view example‑level performance and directly examine errors. Usage analysis over six months shows ModelTracker is frequently used throughout model building, and a controlled experiment demonstrates participants prefer it over traditional tools without compromising model performance.

Abstract

Model building in machine learning is an iterative process. The performance analysis and debugging step typically involves a disruptive cognitive switch from model building to error analysis, discouraging an informed approach to model building. We present ModelTracker, an interactive visualization that subsumes information contained in numerous traditional summary statistics and graphs while displaying example-level performance and enabling direct error examination and debugging. Usage analysis from machine learning practitioners building real models with ModelTracker over six months shows ModelTracker is used often and throughout model building. A controlled experiment focusing on ModelTracker's debugging capabilities shows participants prefer ModelTracker over traditional tools without a loss in model performance.