Multilevel Compression of Random Walks on Networks Reveals Hierarchical Organization in Large Integrated Systems

TLDR

The authors introduce the hierarchical map equation to uncover multilevel structures in large integrated systems. They use an information‑theoretic compression of random‑walker trajectories, finding the shortest multilevel description to yield the optimal hierarchical clustering, and apply a novel search algorithm to reveal rich multilevel organization in large networks. The method reveals country‑level and continental modules in global air traffic, over 100 scientific fields grouped into four major disciplines, shallow hierarchies in globally connected systems, and rich multilevel organization in highly separated networks such as road systems.

Abstract

To comprehend the hierarchical organization of large integrated systems, we introduce the hierarchical map equation, which reveals multilevel structures in networks. In this information-theoretic approach, we exploit the duality between compression and pattern detection; by compressing a description of a random walker as a proxy for real flow on a network, we find regularities in the network that induce this system-wide flow. Finding the shortest multilevel description of the random walker therefore gives us the best hierarchical clustering of the network, the optimal number of levels and modular partition at each level, with respect to the dynamics on the network. With a novel search algorithm, we extract and illustrate the rich multilevel organization of several large social and biological networks. For example, from the global air traffic network we uncover countries and continents, and from the pattern of scientific communication we reveal more than 100 scientific fields organized in four major disciplines: life sciences, physical sciences, ecology and earth sciences, and social sciences. In general, we find shallow hierarchical structures in globally interconnected systems, such as neural networks, and rich multilevel organizations in systems with highly separated regions, such as road networks.