Real-time big data analysis systems resulting from the Internet of Things (IoT)

Abstract

Abstract With the development of advanced telecommunications, the widespread availability of high-speed networks, smartphones, and smart devices, the term "Internet of Things" (IoT) has emerged along with the importance of data generated by it. This importance has increased significantly with the notable increase in the quantity of data, now commonly referred to as "big data". Various studies have emerged that seek to utilize IoT data to drive development in various fields, thus necessitating frameworks that combine big data technologies and IoT services. One of the challenges of dealing with IoT data is its fast flow and the need to collect and process it in real-time. Suitable big data frameworks have been studied to address this challenge. This research compared the Hadoop and Spark systems to select the most appropriate framework for IoT data that meets the requirements for fast flow and analysis, as well as ease of use. The results showed that Spark outperforms Hadoop, particularly in its ability to enable real-time data analysis, speed of processing, and efficient memory usage.