Proceedings of the VLDB Endowment · 2019 · 27 citations · 26 references
ComputingComputer ScienceInformation SystemsData AnalyticsData ManagementWeb AnalyticsDatabase ManagementQuery ManagementQuery OptimizationQuery ProcessingStructured Query LanguageAd Hoc QueryingOn-demand Query ProcessingQuery Performance OptimizationScalable Database DesignScalable Query InfrastructureSql Query LanguagesSystem Use CasesUse Cases
Large organizations like YouTube are dealing with exploding data volume and increasing demand for data driven applications. Broadly, these can be categorized as: reporting and dashboarding, embedded statistics in pages, time-series monitoring, and ad-hoc analysis. Typically, organizations build specialized infrastructure for each of these use cases. This, however, creates silos of data and processing, and results in a complex, expensive, and harder to maintain infrastructure. At YouTube, we solved this problem by building a new SQL query engine - Procella. Procella implements a superset of capabilities required to address all of the four use cases above, with high scale and performance, in a single product. Today, Procella serves hundreds of billions of queries per day across all four workloads at YouTube and several other Google product areas.
26
Sanjay Ghemawat · Communications of the ACM · 2008 · 18.5K citations · Full text
Fay W. Chang, Sanjay Ghemawat, Wilson C. Hsieh et al. · ACM Transactions on Computer Systems · 2008 · 3.4K citations
Luiz André Barroso · Communications of the ACM · 2013 · 1.7K citations
Large-scale cluster management at Google with Borg
Abhishek Verma, Luis Pedrosa, Madhukar Korupolu et al. · 2015 · 1.3K citations · Full text
Michael Armbrust, Reynold Xin, Cheng Lian et al. · 2015 · 1.2K citations