Exploratory data analysis with noisy measurements

TLDR

Multivariate chemical and biological data often exhibit highly heterogeneous measurement error variances, which can arise from the measurements themselves or from preprocessing, and these heteroscedastic errors hinder exploratory data analysis and low‑dimensional visualizations by distorting subspace estimation and contaminating projection spaces. The study proposes a general strategy for exploratory data analysis of multivariate data with highly non‑uniform measurement error variances, leveraging available variance estimates. The strategy comprises three principles: (1) preprocessing with maximum‑likelihood PCA to mitigate noisy measurements; (2) propagating measurement uncertainty through every step; and (3) embedding uncertainty into the visualization via a partial‑transparency projection that modulates object appearance according to measurement quality. The approach’s advantages are demonstrated on simulated data and on experimental yeast microarray gene‑expression time‑course data. © 2012 John Wiley & Sons, Ltd.

Abstract

Multivariate chemical and biological data are increasingly characterized by measurement error variances that are highly heterogeneous. Such heteroscedasticity may be inherent in the measurements themselves or a consequence of data pretreatment. The presence of measurements with large error variances among more precise observations leads to problems in data analysis. For exploratory data analysis and in particular the low‐dimensional visualization of data structures, these complications can result from sources that include preprocessing, subspace estimation, and the projection of objects with erroneous measurements, as well as contamination of the projection space with unreliable samples that preclude the effective visualization of data structures that may be present. In this work, a general strategy is proposed for the exploratory data analysis of multivariate data exhibiting a high degree of non‐uniformity in measurement error variance, where estimates of the variance are available. This strategy involves three principles: (1) mitigation of the effects of noisy measurements through a preprocessing step that uses maximum likelihood principal components analysis; (2) propagation of measurement uncertainty through all steps of the procedure; and (3) incorporation of the uncertainty information into the projection of data onto the visualization subspace. To carry out this last step, a new technique, referred to as the partial transparency projection, is introduced in which the quality of measurements is interactively imbedded into the appearance of the object in the space. The advantages of this strategy are demonstrated with simulated measurements and using experimental microarray gene expression data from a yeast time course study. Copyright © 2012 John Wiley & Sons, Ltd.