Proceedings of the VLDB Endowment · 2011 · 39 citations · 17 references
ComputingComputer ScienceData StructuresDatabasesUncertainty QuantificationNearest Neighbor SearchQuery ProcessingSpatial Database SystemsSpatial DatabasesUncertainty EstimationUncertainty RepresentationIndex SetsIndex StructuresIndex TheoryLower Bound TechniquesNearest Neighbor Data StructuresNearest Neighbor RetrievalStochastic Dominance AnalysisUncertain Databases
Querying uncertain data sets (represented as probability distributions) presents many challenges due to the large amount of data involved and the difficulties comparing uncertainty between distributions. The Earth Mover's Distance (EMD) has increasingly been employed to compare uncertain data due to its ability to effectively capture the differences between two distributions. Computing the EMD entails finding a solution to the transportation problem, which is computationally intensive. In this paper, we propose a new lower bound to the EMD and an index structure to significantly improve the performance of EMD based K-- nearest neighbor (K--NN) queries on uncertain databases. We propose a new lower bound to the EMD that approximates the EMD on a projection vector. Each distribution is projected onto a vector and approximated by a normal distribution, as well as an accompanying error term. We then represent each normal as a point in a Hough transformed space. We then use the concept of stochastic dominance to implement an efficient index structure in the transformed space. We show that our method significantly decreases K--NN query time on uncertain databases. The index structure also scales well with database cardinality. It is well suited for heterogeneous data sets, helping to keep EMD based queries tractable as uncertain data sets become larger and more complex.
17
Optimal aggregation algorithms for middleware
Ronald Fagin, Amnon Lotem, Moni Naor · 2001 · 1.1K citations
Locally adaptive dimensionality reduction for indexing large time series databases
Eamonn Keogh, Kaushik Chakrabarti, Michael J. Pazzani et al. · 2001 · 873 citations
Stochastic Dominance and Expected Utility: Survey and Analysis
Haim Levy · Management Science · 1992 · 853 citations