Exploring Multimodal Data Fusion Through Joint Decompositions with Flexible Couplings

Abstract

A Bayesian framework is proposed to define flexible coupling models for joint tensor decompositions of multiple datasets. Under this framework, a natural formulation of the data fusion problem is to cast it in terms of a joint maximum a posteriori (MAP) estimator. Data-driven scenarios of joint posterior distributions are provided, including general Gaussian priors and non Gaussian coupling priors. We present and discuss implementation issues of algorithms used to obtain the joint MAP estimator. We also show how this framework can be adapted to tackle the problem of joint decompositions of large datasets. In the case of a conditional Gaussian coupling with a linear transformation, we give theoretical bounds on the data fusion performance using the Bayesian Cramér-Rao bound. Simulations are reported for hybrid coupling models ranging from simple additive Gaussian models to Gamma-type models with positive variables and to the coupling of data sets which are inherently of different size due to different resolution of the measurement devices.