The Architecture of PIER: an Internet-Scale Query Processor

Abstract

This paper presents the architecture of PIER, an Internetscale query engine we have been building over the last three years. PIER is the first general-purpose relational query processor targeted at a peer-to-peer (p2p) architecture of thousands or millions of participating nodes on the Internet. It supports massively distributed, database-style dataflows for snapshot and continuous queries. It is intended to serve as a building block for a diverse set of Internet-scale information-centric applications, particularly those that tap into the standardized data readily available on networked machines, including packet headers, system logs, and file names. In earlier