Application of Hadoop Framework in Big Data Analytics

Abstract

Big data is a collection of large data sets that include different types such as structured, unstructured and semi-structured data. This data can be generated from different sources like social media, audios, images, log files, sensor data, transactional applications, web etc. To process or analyse this huge amount of data or extracting meaningful information is a challenging task now a days. Big data exceeds the processing capability of traditional database to capture, manage, and process the voluminous amount of data. In this paper we first introduce the general background of big data and then focus on hadoop platform using map reduce algorithm which provide the environment to implement application in distributed environment. Hadoop is a framework that allows the distributed processing of large data sets. Hadoop is an open source application available under the Apache License. It is designed to scale up from a single server to thousands of machines, where each machine can perform computations locally and store them].