Abstract
AbstractPerformance is a critical concern when reading and writing data from billions of records stored in a Big Data warehouse. We introduce two scopes for query performance improvement. One is to improve the performance of lookup queries after data deletion in Big Data systems that use Eventual Consistency. We propose a scheme to improve lookup performance after data deletion by using Cuckoo Filter. Another scope for improvement is to avoid unnecessary network round-trips for querying in remote nodes in a distributed Big Data cluster when it is known that the nodes do not have requested partition of data. We propose a scheme using probabilistic filters that are looked up before querying remote nodes so that queries resulting in no data can be skipped from passing through the network. We evaluate our schemes with Cassandra using real dataset and show that each scheme can improve performance of lookup queries for up to 2x.
Publisher
Springer Science and Business Media LLC
Subject
Information Systems and Management,Computer Networks and Communications,Hardware and Architecture,Information Systems
Reference39 articles.
1. Vagata P, Wilfong K. Scaling the Facebook data warehouse to 300PB (April 2014). https://code.facebook.com/posts/229861827208629/scaling-the-facebook-data-warehouse-to-300-pb Accessed 30 Dec 2019.
2. Dittrich J, Quiané-Ruiz J-A, Jindal A, Kargin Y, Setty V, Schad J. Hadoop++: Making a yellow elephant run like a cheetah (without it even noticing). Proc VLDB Endow. 2010;3(1–2):515–29. https://doi.org/10.14778/1920841.1920908.
3. Richter S, Quiané-Ruiz J-A, Schuh S, Dittrich J. Towards zero-overhead static and Adaptive Indexing in Hadoop. VLDB J. 2014;23(3):469–94. https://doi.org/10.1007/s00778-013-0332-z.
4. Schuhknecht FM, Dittrich J, Linden L. Adaptive Adaptive Indexing. In: ICDE; 2018.
5. Bloom BH. Space/time trade-offs in hash coding with allowable errors. Commun ACM. 1970;13(7):422–6. https://doi.org/10.1145/362686.362692.
Cited by
6 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献