English

Past, Present and Future of Hadoop: A Survey

Networking and Internet Architecture 2022-03-01 v1
Authors: Ameneh Zarei , Shahla Safari , Mahmood Ahmadi , Farhad Mardukhi
View on arXiv ↗ PDF ↗

Abstract

In this paper, a technology for massive data storage and computing named Hadoop is surveyed. Hadoop consists of heterogeneous computing devices like regular PCs abstracting away the details of parallel processing and developers can just concentrate on their computational problem. A Hadoop cluster is made of two parts: HDFs and Mapreduce. Hadoop cluster uses HDFS for data management. HDFS provides storage for input and output data in MapReduce jobs and is designed with abilities like high-fault tolerance, high-distribution capacity, and high throughput. It is also suitable for storing Terabyte data on clusters and it runs on flexible hardware like commodity devices.

Keywords

hadoop massive data storage mapreduce hdfs distributed computing cluster computing survey

Cite

@article{arxiv.2202.13293,
  title  = {Past, Present and Future of Hadoop: A Survey},
  author = {Ameneh Zarei and Shahla Safari and Mahmood Ahmadi and Farhad Mardukhi},
  journal= {arXiv preprint arXiv:2202.13293},
  year   = {2022}
}

Comments

A survey on Hadoop

Related papers

View all related →
Distributed, Parallel, and Cluster Computing · Computer Science
An Experimental Evaluation of Performance of A Hadoop Cluster on Replica Management
Muralikrishnan Ramane, Sharmila Krishnamoorthy, Sasikala Gowtham
2014-11-10
Distributed, Parallel, and Cluster Computing · Computer Science
Performance Issues of Heterogeneous Hadoop Clusters in Cloud Computing
B. Thirumala Rao, N. V. Sridevi, V. Krishna Reddy, L. S. S. Reddy
2012-07-05
Distributed, Parallel, and Cluster Computing · Computer Science
Management of Data Replication for PC Cluster-based Cloud Storage System
Julia Myint, Thinn Thu Naing
2011-12-30
Distributed, Parallel, and Cluster Computing · Computer Science
Design Architecture-Based on Web Server and Application Cluster in Cloud Environment
Gita Shah, Annappa, K. C. Shet
2014-03-24
Distributed, Parallel, and Cluster Computing · Computer Science
Large-scale Data Modelling in Hive and Distributed Query Processing using MapReduce and Tez
Abzetdin Adamov
2023-01-31
Distributed, Parallel, and Cluster Computing · Computer Science
Optimization and analysis of large scale data sorting algorithm based on Hadoop
Zhuo Wang, Longlong Tian, Dianjie Guo, Xiaoming Jiang
2015-06-02
Distributed, Parallel, and Cluster Computing · Computer Science
Overview of Caching Mechanisms to Improve Hadoop Performance
Rana Ghazali, Douglas G. Down
2023-10-24
Distributed, Parallel, and Cluster Computing · Computer Science
Addressing NameNode Scalability Issue in Hadoop Distributed File System using Cache Approach
Debajyoti Mukhopadhyay, Chetan Agrawal, Devesh Maru, Pooja Yedale +1
2014-11-26
Distributed, Parallel, and Cluster Computing · Computer Science
Beyond Batch Processing: Towards Real-Time and Streaming Big Data
Saeed Shahrivari, Saeed Jalili
2014-08-04
Distributed, Parallel, and Cluster Computing · Computer Science
Integrating R and Hadoop for Big Data Analysis
Bogdan Oancea, Raluca Mariana Dragoescu
2018-02-09
Distributed, Parallel, and Cluster Computing · Computer Science
Building and Installing a Hadoop/MapReduce Cluster from Commodity Components
Jochen L. Leidner, Gary Berosik
2009-12-01
Distributed, Parallel, and Cluster Computing · Computer Science
Performance Evaluation of Virtualized Hadoop Clusters
Todor Ivanov, Roberto V. Zicari, Sead Izberovic, Karsten Tolle
2014-11-17
Distributed, Parallel, and Cluster Computing · Computer Science
Big Data Analytics in Cloud environment using Hadoop
Mansaf Alam, Kashish Ara Shakil
2016-10-17
Distributed, Parallel, and Cluster Computing · Computer Science
Data-Intensive Workload Consolidation on Hadoop Distributed File System
Reza Moraveji, Javid Taheri, MohammadReza HosseinyFarahabady, Nikzad Babaii Rizvandi +1
2016-11-15
Distributed, Parallel, and Cluster Computing · Computer Science
Lustre, Hadoop, Accumulo
Jeremy Kepner, William Arcand, David Bestor, Bill Bergeron +10
2016-06-21
Databases · Computer Science
Analyzing Large-Scale, Distributed and Uncertain Data
Yaron Gonen
2017-12-06
Distributed, Parallel, and Cluster Computing · Computer Science
Sector and Sphere: Towards Simplified Storage and Processing of Large Scale Distributed Data
Yunhong Gu, Robert L Grossman
2009-01-17
Distributed, Parallel, and Cluster Computing · Computer Science
Parallel Spectral Clustering Algorithm Based on Hadoop
Yajun Cui, Yang Zhao, Kafei Xiao, Chenglong Zhang +1
2015-06-02
Networking and Internet Architecture · Computer Science
A Survey of Big Data Machine Learning Applications Optimization in Cloud Data Centers and Networks
Sanaa Hamid Mohamed, Taisir E. H. El-Gorashi, Jaafar M. H. Elmirghani
2019-10-03
Distributed, Parallel, and Cluster Computing · Computer Science
Survey on Improved Scheduling in Hadoop MapReduce in Cloud Environments
B. Thirumala Rao, L. S. S. Reddy
2012-07-04
Distributed, Parallel, and Cluster Computing · Computer Science
Hadoop Perfect File: A fast access container for small files with direct in disc metadata access
Jude Tchaye-Kondi, Yanlong Zhai, Kwei-Jay Lin, Wenjun Tao +1
2021-06-15
Distributed, Parallel, and Cluster Computing · Computer Science
Advancing Polyglot Big Data Processing using the Hadoop ecosystem
Antony Seabra, Sergio Lifschitz
2025-04-22
Distributed, Parallel, and Cluster Computing · Computer Science
Storage Solutions for Big Data Systems: A Qualitative Study and Comparison
Samiya Khan, Xiufeng Liu, Syed Arshad Ali, Mansaf Alam
2019-04-29
Distributed, Parallel, and Cluster Computing · Computer Science
Efficient Support of Big Data Storage Systems on the Cloud
Akshay MS, Suhas Mohan, Vincent Kuri, Dinkar Sitaram +1
2014-12-01
Distributed, Parallel, and Cluster Computing · Computer Science
Data Mining Using High Performance Data Clouds: Experimental Studies Using Sector and Sphere
Robert L Grossman, Yunhong Gu
2008-08-25