Probabilistic programming in Python using PyMC3

TLDR

Probabilistic programming enables automatic Bayesian inference on user-defined models, and recent advances in Hamiltonian Monte Carlo have expanded its applicability, though gradient requirements often limit use; PyMC3 distinguishes itself by allowing model specification directly in Python, providing flexibility without a domain‑specific language. The paper provides a tutorial‑style introduction to the PyMC3 software package. PyMC3 is an open‑source Python framework that uses Theano for automatic differentiation of gradients and compiles probabilistic programs to C for speed.

Abstract

Probabilistic Programming allows for automatic Bayesian inference on user-defined probabilistic models. Recent advances in Markov chain Monte Carlo (MCMC) sampling allow inference on increasingly complex models. This class of MCMC, known as Hamliltonian Monte Carlo, requires gradient information which is often not readily available. PyMC3 is a new open source Probabilistic Programming framework written in Python that uses Theano to compute gradients via automatic differentiation as well as compile probabilistic programs on-the-fly to C for increased speed. Contrary to other Probabilistic Programming languages, PyMC3 allows model specification directly in Python code. The lack of a domain specific language allows for great flexibility and direct interaction with the model. This paper is a tutorial-style introduction to this software package.