Modeling distances in large-scale networks by matrix factorization

Abstract

In this paper, we propose a model for representing and predicting distances in large-scale networks by <i>matrix factorization</i>. The model is useful for network distance sensitive applications, such as content distribution networks, topology-aware overlays, and server selections. Our approach overcomes several limitations of previous coordinates-based mechanisms, which cannot model sub-optimal routing or asymmetric routing policies. We describe two algorithms --- singular value decomposition (SVD) and nonnegative matrix factorization (NMF)---for representing a matrix of network distances as the product of two smaller matrices. With such a representation, we build a scalable system--- <i>Internet Distance Estimation Service (IDES)</i>---that predicts large numbers of network distances from limited numbers of measurements. Extensive simulations on real-world data sets show that IDES leads to more accurate, efficient and robust predictions of latencies in large-scale networks than previous approaches.