Affiliation:
1. Department of Electrical and Computer Engineering, Western University, London, ON N6A 5B9, Canada
Abstract
This paper presents a comprehensive literature review on the evolution of data-lake technology, with a particular focus on data-lake architectures. By systematically examining the existing body of research, we identify and classify the major types of data-lake architectures that have been proposed and implemented over time. The review highlights key trends in the development of data-lake architectures, identifies the primary challenges faced in their implementation, and discusses future directions for research and practice in this rapidly evolving field. We have developed diagrammatic representations to highlight the evolution of various architectures. These diagrams use consistent notations across all architectures to further enhance the comparative analysis of the different architectural components. We also explore the differences between data warehouses and data lakes. Our findings provide valuable insights for researchers and practitioners seeking to understand the current state of data-lake technology and its potential future trajectory.
Funder
Libyan Ministry of Higher Education and Scientific Research
Reference77 articles.
1. Naeem, M., Jamal, T., Diaz-Martinez, J., Butt, S.A., Montesano, N., Tariq, M.I., De-la Hoz-Franco, E., and De-La-Hoz-Valdiris, E. (2022). Trends and future perspective challenges in big data. Advances in Intelligent Data Analysis and Applications, Proceeding of the Sixth Euro-China Conference on Intelligent Data Analysis and Applications, Arad, Romania, 15–18 October 2019, Springer.
2. (2024, May 27). Data Growth Worldwide 2010–2025|Statista. Available online: https://www.statista.com/statistics/871513/worldwide-data-created/.
3. Critical analysis of Big Data challenges and analytical methods;Sivarajah;J. Bus. Res.,2017
4. John, T., and Misra, P. (2017). Data Lake for Enterprises, Packt Publishing Ltd.
5. LaPlante, A. (2016). Architecting Data Lakes, O’Reilly Media.