Author:
Tembo Welo Audrey,Lubaki Kinzonzi Hervé,Bila Khonde Noel,Mbuyi Mukendi Eugène
Abstract
Nowadays, the processing of big data has become a major preoccupation for businesses, not only for storage and processing but also for operational requirements such as speed, maintaining performance with scalability, reliability, availability, security, and cost control; ultimately enabling them to maximize their profits by using the new possibilities offered by Big Data. In this article, we will explore and exploit the concepts and architectures of Big Data, in particular through the Hadoop open-source framework, and see how it meets the needs set out above, in its cluster structure, its components, its Lambda and Kappa architectures, and so on. We are also going to deploy Hadoop in a virtualized Linux environment, with several nodes, under the Oracle Virtual Box virtualization software, and use the experimental method to compare the processing time of the MapReduce algorithm on two DataSets with successively one, two, and three and four Datanodes, and thus observe the gains in processing time with the increase in the number of nodes in the cluster
Publisher
Politeknik Negeri Cilacap
Reference23 articles.
1. J. B. N. Penka, S. Mahmoudi, and O. Debauche, "A new Kappa Architecture for IoT Data Management in Smart Farming," in The 18th International Conference on Mobile Systems and Pervasive Computing (MobiSPC), Leuven, Belgium, Aug. 9-12, 2021, Procedia Computer Science, Sep. 2021.
2. G. K. Kalipe and R. K. Behera, "Big Data Architectures: A Detailed and Application Oriented Analysis," International Journal of Innovative Technology and Exploring Engineering (IJITEE), vol. 8, no. 9, Jul. 2019, ISSN: 2278-3075.
3. J. Lin, "The Lambda and the Kappa," University of Waterloo, Sep./Oct. 2017, IEEE Internet Computing.
4. Mr. H. Hashem, "Modélisation intégratrice du traitement BigData," Thèse de doctorat, Télécom SudParis, Ecole doctorale STIC, Université Paris-Saclay, Evry, France, Sep. 19, 2016.
5. A. Gillet, É. Leclercq, and N. Cullot, "Évolution et formalisation de la Lambda Architecture pour des analyses à hautes performances - Application aux données de Twitter," 2021 ISTE OpenScience, Published by ISTE Ltd., London, UK, openscience.fr.