2019 · 12 citations · 23 references
ComputingComputer ScienceComputer SystemsData ManagementDatabasesDistributed SystemsDistributed Data ProcessingDistributed Geospatial DatabasesDistributed StorageHigh-throughput ComputingQuery ProcessingSpatial Database SystemsBig Data ArchitectureData PlacementIndexingLocation-based Query ProcessingMapreducePartitioned Global Address SpaceQuery Latency OptimizationSpatial Data Management
Location-based services (LBS) are a widely adopted technology that produces large volumes of spatio-temporal data at high velocity. Spatial data is also being generated from many other geo-spatial applications. To address the challenge of data volume, a number of big spatial data management systems have emerged that are based on the MapReduce paradigm. Recent projects have developed spatial data systems using Spark's distributed in-memory architecture. These projects, which include GeoSpark, SpatialSpark, and LocationSpark, do not support the high update rates required by LBS applications. Alternatively, systems such as MD-HBase support data updates, but are hindered by the performance characteristics of HBase, which is a disk-oriented framework. We present DISTIL+, a distributed spatio-temporal data processing system designed for high velocity location data. Our system achieves high update throughput and low query latency by leveraging the APGAS (Asynchronous Partitioned Global Address Space) architecture to build a multi-level distributed in-memory index. We present extensive experimental evaluation of our system, comparing several indexing and data placement schemes, as well as competing systems. Our results show that DISTIL+ excels at supporting high throughput location updates, and low latency spatio-temporal range queries and kNN queries, while offering better performance than existing approaches.
23
Sanjay Ghemawat · Communications of the ACM · 2008 · 18.5K citations · Full text
The Hadoop Distributed File System
Konstantin V. Shvachko, Hairong Kuang, Sanjay Radia et al. · 2010 · 4.8K citations
Spark: cluster computing with working sets
Matei Zaharia, Mosharaf Chowdhury, Michael J. Franklin et al. · 2010 · 4.2K citations
The log-structured merge-tree (LSM-tree)
Patrick E. O׳Neil, E.T. Cheng, Dieter Gawlick et al. · Acta Informatica · 1996 · 1.3K citations
Philippe Charles, Christian Grothoff, Vijay Saraswat et al. · 2005 · 1.3K citations