Scaling of Streaming Data Using Machine Learning Algorithms

Abstract

Today, data is generated continuously by millions of data sources, which send in the records simultaneously, in small to large sizes. The rapid growth of data in velocity, volume, value, variety, and veracity has presented big challenges for businesses of all types. This type of data is called streaming data. Streaming data includes a variety of data such as mobile application notifications, e-commerce purchases, sensors in transportation vehicles, information from social applications, IoT sensors. This data is required to be processed sequentially and incrementally on record by record and used for a wide variety of analytics including correlations, filtering, and sampling. Information derived from such analysis gives visibility into many aspects such as customer activity, website clicks, geo-location of devices. There has been a great interest in developing systems for processing continuous data streams. This chapter aims to design a scalable system that can instantly analyze the data using machine learning algorithms.