Abstract
Traditional data analytics tools are designed to deal with the asymmetrical type of data i.e., structured, semi-structured, and unstructured. The diverse behavior of data produced by different sources requires the selection of suitable tools. The restriction of recourses to deal with a huge volume of data is a challenge for these tools, which affects the performances of the tool’s execution time. Therefore, in the present paper, we proposed a time optimization model, shares common HDFS (Hadoop Distributed File System) between three Name-node (Master Node), three Data-node, and one Client-node. These nodes work under the DeMilitarized zone (DMZ) to maintain symmetry. Machine learning jobs are explored from an independent platform to realize this model. In the first node (Name-node 1), Mahout is installed with all machine learning libraries through the maven repositories. The second node (Name-node 2), R connected to Hadoop, is running through the shiny-server. Splunk is configured in the third node (Name-node 3) and is used to analyze the logs. Experiments are performed between the proposed and legacy model to evaluate the response time, execution time, and throughput. K-means clustering, Navies Bayes, and recommender algorithms are run on three different data sets, i.e., movie rating, newsgroup, and Spam SMS data set, representing structured, semi-structured, and unstructured data, respectively. The selection of tools defines data independence, e.g., Newsgroup data set to run on Mahout as others cannot be compatible with this data. It is evident from the outcome of the data that the performance of the proposed model establishes the hypothesis that our model overcomes the limitation of the resources of the legacy model. In addition, the proposed model can process any kind of algorithm on different sets of data, which resides in its native formats.
Subject
Physics and Astronomy (miscellaneous),General Mathematics,Chemistry (miscellaneous),Computer Science (miscellaneous)
Reference50 articles.
1. Big-data computing: Creating revolutionary breakthroughs in commerce;Bryant;Sci. Soc.,2008
2. Extracting value from chaos;Gantz;IDC Iview,2011
3. A Multiple-Kernel Fuzzy C-Means Algorithm for Image Segmentation
4. Beyond the hype: Big data concepts, methods, and analytics
Cited by
39 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. An Intelligent Approach to Grape Leaf Disease Diagnosis Through Machine Learning;2024 2nd International Conference on Disruptive Technologies (ICDT);2024-03-15
2. Detection Of Breast Tumour Using Advanced Methods;2024 2nd International Conference on Disruptive Technologies (ICDT);2024-03-15
3. Classification of Sign Language Gestures using CNN with Adam Optimizer;2024 2nd International Conference on Disruptive Technologies (ICDT);2024-03-15
4. A Cyber Physical Security for Electrical Vehicles using Deep learning;2024 International Conference on Automation and Computation (AUTOCOM);2024-03-14
5. A Multi-Source Information Retrieval System for Question Answering with Sentiment Analysis and LLM Interpretation;2024 International Conference on Automation and Computation (AUTOCOM);2024-03-14